feat(lzendpoint): Add date/listened during MS specific properties

#360
feat(lzendpoint): Add MS specific timestamp SOC to payload
2026-09-03 05:10:00 +03:00 · 2025-09-19 13:06:23 +00:00 · 2025-09-19 12:54:20 +00:00 · 2025-09-18 14:15:20 +00:00 · 2025-09-18 13:45:29 +00:00 · 2025-09-17 08:54:44 -04:00
40 changed files with 1929 additions and 290 deletions
@@ -0,0 +1,25 @@
+version: '3.7'
+services:
+  # Update this to the name of the service you want to work with in your docker-compose.yml file
+  app:
+    # Uncomment if you want to override the service's Dockerfile to one in the .devcontainer 
+    # folder. Note that the path of the Dockerfile and context is relative to the *primary* 
+    # docker-compose.yml file (the first in the devcontainer.json "dockerComposeFile"
+    # array). The sample below assumes your primary file is in the root of your project.
+    #
+    # build:
+    #   context: .
+    #   dockerfile: .devcontainer/Dockerfile
+
+    image: mcr.microsoft.com/devcontainers/javascript-node:1-20-bookworm
+
+    volumes:
+      # Update this to wherever you want VS Code to mount the folder of your project
+      - ..:/workspaces:cached
+    command: sleep infinity
+
+  valkey:
+    image: valkey/valkey
+    env_file:
+      - path: ./.env
+        required: false
@@ -3,7 +3,13 @@
 {
 	"name": "Node.js",
 	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
-	"image": "mcr.microsoft.com/devcontainers/javascript-node:1-20-bookworm",
+	"dockerComposeFile": [
+		"compose.yml"
+	],
+
+	"service": "app",
+
+	"workspaceFolder": "/workspaces",

 	// Features to add to the dev container. More info: https://containers.dev/features.
 	// "features": {},
@@ -118,7 +118,9 @@ dist
 .pnp.*

 config/*.json
+config/mscache
 config/yti-*
+config/*.cache
 *.txt
 .idea/

@@ -3,5 +3,8 @@
  "extension": "ts",
  "import": "tsx/esm",
  "spec": "./src/backend/tests/**/*.test.ts",
+  "file": [
+    "./src/backend/tests/setup.ts"
+  ],
  "exit": true
 }
@@ -186,6 +186,93 @@ Useful when running with [docker](../installation/installation.mdx#docker) so th

 </details>

+#### Caching
+
+Multi-scrobbler can cache some of its data for the purpose of **persisting queued and dead scrobbles** after restarting.
+
+It supports caching using either:
+
+* **File** cache stored in the `CONFIG_DIR` directory (next to your File/[AIO](./?configType=aio#configuration-types) file)
+* [**Valkey**](https://valkey.io/), an open-source fork of Redis.
+
+**File** caching is **enabled by default** when no other configuration is present.
+
+The type of cache used, and it's connection properties, can be configured through ENV or **AIO** config.
+
+##### Caching Configuration
+
+<Tabs groupId="cache" queryString>
+
+<TabItem value="file" label="File">
+
+**File** cache is stored in the `CONFIG_DIR` directory using the pre-defined file name `ms-scrobble.cache`.
+
+It is **enabled by default** if `CACHE_SCROBBLE`, or the respective AIO configuration, is not defined.
+
+<Tabs groupId="configType" queryString>
+
+<TabItem value="env" label="ENV">
+
+| Environmental Variable | Required? | Default              | Description                                                  |
+| :--------------------- | --------- | -------------------- | ------------------------------------------------------------ |
+| `CACHE_SCROBBLE`       | No        | `file`               | The cache type to use                                        |
+| `CACHE_SCROBBLE_CONN`  | No        | The config directory | The directory, within the container, to store the cache file |
+
+</TabItem>
+
+<TabItem value="aio" label="AIO">
+
+```json5 title="config.json"
+{
+    "cache": {
+        "scrobble": {
+            "provider": "file",
+            "connection": "/config"
+        }
+    },
+    // ...
+}
+```
+
+</TabItem>
+
+</Tabs>
+</TabItem>
+
+<TabItem value="valkey" label="Valkey">
+
+<Tabs groupId="configType" queryString>
+
+<TabItem value="env" label="ENV">
+
+| Environmental Variable | Required? | Default              | Description                                                  |
+| :--------------------- | --------- | -------------------- | ------------------------------------------------------------ |
+| `CACHE_SCROBBLE`       | Yes        | `valkey`               | The cache type to use                                      |
+| `CACHE_SCROBBLE_CONN`  | Yes        |  | The host/IP and port to connect to, prefixed with `redis://` -- EX: `redis://192.168.0.120:6379` |
+
+</TabItem>
+
+<TabItem value="aio" label="AIO">
+
+```json5 title="config.json"
+{
+    "cache": {
+        "scrobble": {
+            "provider": "valkey",
+            "connection": "redis://192.168.0.120:6379"
+        }
+    },
+    // ...
+}
+```
+
+</TabItem>
+
+</Tabs>
+</TabItem>
+
+</Tabs>
+
 #### Debug Mode

 Turning on Debug Mode will
@@ -2358,17 +2445,12 @@ Due to monitoring being unofficial, listening history from YTM can be inconsiste

 :::

-#### Authentication
+#### Authentication {#ytm-auth}

-Only one of these methods needs to be used. **Cookies** are easier but **OAuth Client**  may be more stable.
+Only one of these methods needs to be used.

 <Tabs groupId="ytmAuth" queryString>
    <TabItem value="cookie" label="Cookies">
-        :::info
-
-        If cookies stop working for you or are being invalidated often try switching to **OAuth Client** authentication.
-
-        :::

        Use instructions from 
        
@@ -2400,10 +2482,14 @@ Only one of these methods needs to be used. **Cookies** are easier but **OAuth C

    </TabItem>
    <TabItem value="oauth" label="OAuth Client">
-        :::note
+        :::warning
+
+        As of Sept 25' OAuth clients no longer seem to have permission to access the YTM service, as reported by the upstream YTM library developer and [several other users.](https://github.com/FoxxMD/multi-scrobbler/issues/345#issuecomment-3258518769)
+
+        If you have existing, working OAuth credentials keep using them but there is no gaurantee they will continue to work.
+
+        If you are setting up the YTM Source for the first time you should first try to use [Cookies](./?ytmAuth=cookie#ytm-auth).

-        This is likely to be the most stable method and least likely to be blocked or have authentication invalidated after an extended period. It requires more setup but is worth the effort.
-        
        :::

        [Based on the instructions from here...](https://github.com/LuanRT/YouTube.js/issues/803#issuecomment-2479689924)
@@ -19,12 +19,13 @@
        "@fortawesome/react-fontawesome": "^0.2.0",
        "@foxxmd/chromecast-client": "^1.0.4",
        "@foxxmd/get-version": "^0.0.3",
-        "@foxxmd/logging": "^0.2.2",
+        "@foxxmd/logging": "^0.2.3",
+        "@foxxmd/redact-string": "^0.1.2",
        "@foxxmd/regex-buddy-core": "^0.1.2",
        "@foxxmd/string-sameness": "^0.4.0",
-        "@gr2m/net-interceptor": "^1.0.0",
        "@jellyfin/sdk": "^0.11.0",
        "@kenyip/backoff-strategies": "^1.0.4",
+        "@keyv/valkey": "^1.0.8",
        "@lukehagar/plexjs": "^0.39.0",
        "@react-nano/use-event-source": "^0.13.0",
        "@reduxjs/toolkit": "^1.9.5",
@@ -36,7 +37,9 @@
        "avahi-browse": "^1.1.4",
        "better-sse": "^0.8.0",
        "body-parser": "^1.19.0",
+        "cacheable": "^1.10.4",
        "castv2": "^0.1.10",
+        "clone": "^2.1.2",
        "clsx": "^2.0.0",
        "common-tags": "^1.8.2",
        "compare-versions": "^4.1.2",
@@ -49,6 +52,7 @@
        "express-session": "^1.17.2",
        "fast-deep-equal": "^3.1.3",
        "fixed-size-list": "^0.3.0",
+        "flat-cache": "^6.1.13",
        "formidable": "^3.5",
        "glob": "^11.0.0",
        "google-auth-library": "^9.15.0",
@@ -56,15 +60,16 @@
        "http-cookie-agent": "^7.0.1",
        "icecast-metadata-stats": "^0.1.12",
        "iso-websocket": "^0.3.0",
-        "iti": "^0.6.0",
+        "iti": "^0.7.0",
        "json5": "^2.2.3",
+        "keyv": "^5.5.0",
        "kodi-api": "^0.2.1",
        "lastfm-node-client": "^2.2.0",
        "mopidy": "^1.3.0",
        "mpd-api": "^1.1.2",
        "nanoid": "^3.3.1",
        "normalize-url": "^8.0.1",
-        "ntfy": "^1.6.2",
+        "ntfy": "^1.7.6",
        "object-hash": "^3.0.0",
        "p-event": "^4.2.0",
        "passport": "^0.6.0",
@@ -83,6 +88,8 @@
        "tough-cookie": "^5.1.2",
        "ts-json-schema-generator": "^2.3.0",
        "tsx": "^4.7.0",
+        "typeson": "^9.0.4",
+        "typeson-registry": "^11.1.1",
        "vite-express": "^0.16.0",
        "vlc-client": "^1.1.1",
        "xml2js": "0.6.1",
@@ -92,6 +99,7 @@
        "@dbus-types/notifications": "^0.0.5",
        "@eslint/js": "^8.56.0",
        "@faker-js/faker": "^9.0.1",
+        "@gr2m/net-interceptor": "^1.0.0",
        "@istanbuljs/nyc-config-typescript": "^1.0.2",
        "@testing-library/jest-dom": "^5.17.0",
        "@testing-library/react": "^13.4.0",
@@ -121,7 +129,6 @@
        "@vitejs/plugin-react": "^4.2.1",
        "chai": "^4.3.6",
        "chai-as-promised": "^7.1.1",
-        "clone": "^2.1.2",
        "eslint": "^8.56.0",
        "eslint-plugin-prefer-arrow-functions": "^3.2.4",
        "mocha": "^10.3.0",
@@ -132,7 +139,7 @@
        "typescript": "5.5.4",
        "typescript-eslint": "^7.0.1",
        "vite": "^5.2.12",
-        "with-local-tmp-dir": "^5.1.1"
+        "with-local-tmp-dir": "^6.0.0"
      },
      "engines": {
        "node": ">=18.19.1",
@@ -554,12 +561,13 @@
      "integrity": "sha512-lp316jRb+OCCm8KbzidppWCKteISwgpWU6sxzzDI+PJtflfX87h/AHVQdcqY1f3/za2S869KliJbtJZuan1H2Q=="
    },
    "node_modules/@dword-design/chdir": {
-      "version": "2.1.4",
-      "resolved": "https://registry.npmjs.org/@dword-design/chdir/-/chdir-2.1.4.tgz",
-      "integrity": "sha512-cHpMTx4XepHQi3hmDnKma5YXU4pm01k9xKI1k8Wwq25cN3j0FCWbiji11hdrRYr6+uDC9mrb/M5Mo0okYiNRzA==",
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/@dword-design/chdir/-/chdir-4.0.0.tgz",
+      "integrity": "sha512-Lw0SDZbeMnsGwBwZNeazDh5RL9oVXsEVD9M3S2hOrgQTNq3LwCs7ndxO72exOGqC+OVQBt8OFQ08gsUOX7mdCQ==",
      "dev": true,
+      "license": "MIT",
      "engines": {
-        "node": ">=12"
+        "node": ">=20"
      },
      "funding": {
        "url": "https://github.com/sponsors/dword-design"
@@ -1117,9 +1125,10 @@
      }
    },
    "node_modules/@foxxmd/logging": {
-      "version": "0.2.2",
-      "resolved": "https://registry.npmjs.org/@foxxmd/logging/-/logging-0.2.2.tgz",
-      "integrity": "sha512-aEbLh6kDqr5UJ1yBl/h/31bE2XyTfQob6EkgYBVaJb/T+qaVeB/T7T3nfKLZQhS31Y53/LxQKrwHU30ZkkBv4Q==",
+      "version": "0.2.3",
+      "resolved": "https://registry.npmjs.org/@foxxmd/logging/-/logging-0.2.3.tgz",
+      "integrity": "sha512-KMqWy42niMLMyz8YmQ09XVay5szhU2Y8lWzM3Fbm+zyfFRTiaYCSal5tQSeWmVdjps6LqtQNGDH4KG5a3Wgnrw==",
+      "license": "MIT",
      "dependencies": {
        "pino": "^9.2.0",
        "pino-abstract-transport": "^1.2.0",
@@ -1132,6 +1141,16 @@
        "npm": ">=9.3.0"
      }
    },
+    "node_modules/@foxxmd/redact-string": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/@foxxmd/redact-string/-/redact-string-0.1.2.tgz",
+      "integrity": "sha512-/YTLauymSs2AdlxtwWE86uk3kwQB0v/r7KzFRlnCojPjIdVzbiGxDPYuRxwfYY9hEg/8lRa47MbaYN2v4BqgTw==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=16.20.0",
+        "npm": ">=8.19.4"
+      }
+    },
    "node_modules/@foxxmd/regex-buddy-core": {
      "version": "0.1.2",
      "resolved": "https://registry.npmjs.org/@foxxmd/regex-buddy-core/-/regex-buddy-core-0.1.2.tgz",
@@ -1162,6 +1181,7 @@
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/@gr2m/net-interceptor/-/net-interceptor-1.0.0.tgz",
      "integrity": "sha512-qLTp3npmB5SDiEIh1Jg1v8a09dlnfYV1kcSyLmsyucvZBT89nM+6L1QdlrCC/W1cTvdTpvI+Vleit94UVX7/Cg==",
+      "dev": true,
      "engines": {
        "node": ">= 14"
      }
@@ -1305,6 +1325,12 @@
        "@types/node": ">=18"
      }
    },
+    "node_modules/@iovalkey/commands": {
+      "version": "0.1.0",
+      "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",
+      "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",
+      "license": "MIT"
+    },
    "node_modules/@isaacs/cliui": {
      "version": "8.0.2",
      "resolved": "https://registry.npmjs.org/@isaacs/cliui/-/cliui-8.0.2.tgz",
@@ -1533,6 +1559,24 @@
      "resolved": "https://registry.npmjs.org/@kenyip/backoff-strategies/-/backoff-strategies-1.0.4.tgz",
      "integrity": "sha512-vduQZw2ctS3kIuSnCSSRiE4J90Y8WShR9xVG+e1lvFWksU2aTxjdkArcQqJ+XLm22JS380OZmrIPY1U06TAsng=="
    },
+    "node_modules/@keyv/serialize": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/@keyv/serialize/-/serialize-1.1.0.tgz",
+      "integrity": "sha512-RlDgexML7Z63Q8BSaqhXdCYNBy/JQnqYIwxofUrNLGCblOMHp+xux2Q8nLMLlPpgHQPoU0Do8Z6btCpRBEqZ8g==",
+      "license": "MIT"
+    },
+    "node_modules/@keyv/valkey": {
+      "version": "1.0.8",
+      "resolved": "https://registry.npmjs.org/@keyv/valkey/-/valkey-1.0.8.tgz",
+      "integrity": "sha512-V2GziCVL5xpeVKrOo0EztKErQ+N9ZQ+a1ROPoxCU7AQ/aSIa7oRaQZzN2kOOFG6upTKLYeiUFz2q5vjacMEeIg==",
+      "license": "MIT",
+      "dependencies": {
+        "iovalkey": "^0.3.3"
+      },
+      "engines": {
+        "node": ">= 18"
+      }
+    },
    "node_modules/@lukehagar/plexjs": {
      "version": "0.39.0",
      "resolved": "https://registry.npmjs.org/@lukehagar/plexjs/-/plexjs-0.39.0.tgz",
@@ -2736,6 +2780,17 @@
        "@types/node": "*"
      }
    },
+    "node_modules/@types/fs-extra": {
+      "version": "11.0.4",
+      "resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-11.0.4.tgz",
+      "integrity": "sha512-yTbItCNreRooED33qjunPthRcSjERP1r4MqCZc7wv0u2sUkzTFp45tgUfS5+r7FrZPdmCCNflLhVSP/o+SemsQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/jsonfile": "*",
+        "@types/node": "*"
+      }
+    },
    "node_modules/@types/hoist-non-react-statics": {
      "version": "3.3.5",
      "resolved": "https://registry.npmjs.org/@types/hoist-non-react-statics/-/hoist-non-react-statics-3.3.5.tgz",
@@ -2781,6 +2836,16 @@
      "resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.15.tgz",
      "integrity": "sha512-5+fP8P8MFNC+AyZCDxrB2pkZFPGzqQWUzpSeuuVLvm8VMcorNYavBqoFcxK8bQz4Qsbn4oUEEem4wDLfcysGHA=="
    },
+    "node_modules/@types/jsonfile": {
+      "version": "6.1.4",
+      "resolved": "https://registry.npmjs.org/@types/jsonfile/-/jsonfile-6.1.4.tgz",
+      "integrity": "sha512-D5qGUYwjvnNNextdU59/+fI+spnwtTFmyQP0h+PfIOSkNfpU6AOICUOkm4i0OnSk+NyjdPJrxCDro0sJsWlRpQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/node": "*"
+      }
+    },
    "node_modules/@types/keyv": {
      "version": "3.1.4",
      "resolved": "https://registry.npmjs.org/@types/keyv/-/keyv-3.1.4.tgz",
@@ -2890,6 +2955,13 @@
        "@types/node": "*"
      }
    },
+    "node_modules/@types/retry": {
+      "version": "0.12.2",
+      "resolved": "https://registry.npmjs.org/@types/retry/-/retry-0.12.2.tgz",
+      "integrity": "sha512-XISRgDJ2Tc5q4TRqvgJtzsRkFYNJzZrhTdtMoGVBttwzzQJkPnS3WWTFc7kuDRoPtPakl+T+OfdEUjYJj7Jbow==",
+      "dev": true,
+      "license": "MIT"
+    },
    "node_modules/@types/send": {
      "version": "0.17.4",
      "resolved": "https://registry.npmjs.org/@types/send/-/send-0.17.4.tgz",
@@ -3587,13 +3659,13 @@
      }
    },
    "node_modules/axios": {
-      "version": "1.7.7",
-      "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.7.tgz",
-      "integrity": "sha512-S4kL7XrjgBmvdGut0sN3yJxqYzrDOnivkBiN0OFs6hLiUam3UPvswUo0kqGyhqUZGEOytHyumEdXsAkgCOUf3Q==",
-      "peer": true,
+      "version": "1.11.0",
+      "resolved": "https://registry.npmjs.org/axios/-/axios-1.11.0.tgz",
+      "integrity": "sha512-1Lx3WLFQWm3ooKDYZD1eXmoGO9fxYQjrycfHFC8P0sCfQVXyROp0p9PFWBehewBOdCwHc+f/b8I0fMto5eSfwA==",
+      "license": "MIT",
      "dependencies": {
        "follow-redirects": "^1.15.6",
-        "form-data": "^4.0.0",
+        "form-data": "^4.0.4",
        "proxy-from-env": "^1.1.0"
      }
    },
@@ -3602,6 +3674,15 @@
      "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
      "integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="
    },
+    "node_modules/base64-arraybuffer-es6": {
+      "version": "3.1.0",
+      "resolved": "https://registry.npmjs.org/base64-arraybuffer-es6/-/base64-arraybuffer-es6-3.1.0.tgz",
+      "integrity": "sha512-QKKtftiSrKjilihGNLXxnrb9LJj7rnEdB1cYAqVpekFy0tisDklAf1RAgvpm0HsGYx9sv7FUbgpsrfwTyCPVLg==",
+      "license": "MIT",
+      "engines": {
+        "node": "^12.22.0 || ^14.17.0 || >=16.0.0"
+      }
+    },
    "node_modules/base64-js": {
      "version": "1.5.1",
      "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
@@ -3791,6 +3872,16 @@
        "node": ">= 0.8"
      }
    },
+    "node_modules/cacheable": {
+      "version": "1.10.4",
+      "resolved": "https://registry.npmjs.org/cacheable/-/cacheable-1.10.4.tgz",
+      "integrity": "sha512-Gd7ccIUkZ9TE2odLQVS+PDjIvQCdJKUlLdJRVvZu0aipj07Qfx+XIej7hhDrKGGoIxV5m5fT/kOJNJPQhQneRg==",
+      "license": "MIT",
+      "dependencies": {
+        "hookified": "^1.11.0",
+        "keyv": "^5.5.0"
+      }
+    },
    "node_modules/cacheable-lookup": {
      "version": "2.0.1",
      "resolved": "https://registry.npmjs.org/cacheable-lookup/-/cacheable-lookup-2.0.1.tgz",
@@ -3803,6 +3894,15 @@
        "node": ">=10"
      }
    },
+    "node_modules/cacheable-lookup/node_modules/keyv": {
+      "version": "4.5.4",
+      "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz",
+      "integrity": "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==",
+      "license": "MIT",
+      "dependencies": {
+        "json-buffer": "3.0.1"
+      }
+    },
    "node_modules/cacheable-request": {
      "version": "7.0.4",
      "resolved": "https://registry.npmjs.org/cacheable-request/-/cacheable-request-7.0.4.tgz",
@@ -3820,6 +3920,15 @@
        "node": ">=8"
      }
    },
+    "node_modules/cacheable-request/node_modules/keyv": {
+      "version": "4.5.4",
+      "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz",
+      "integrity": "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==",
+      "license": "MIT",
+      "dependencies": {
+        "json-buffer": "3.0.1"
+      }
+    },
    "node_modules/cacheable-request/node_modules/normalize-url": {
      "version": "6.1.0",
      "resolved": "https://registry.npmjs.org/normalize-url/-/normalize-url-6.1.0.tgz",
@@ -3865,6 +3974,19 @@
        "url": "https://github.com/sponsors/ljharb"
      }
    },
+    "node_modules/call-bind-apply-helpers": {
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
+      "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
+      "license": "MIT",
+      "dependencies": {
+        "es-errors": "^1.3.0",
+        "function-bind": "^1.1.2"
+      },
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
    "node_modules/callsites": {
      "version": "3.1.0",
      "resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
@@ -4104,7 +4226,7 @@
      "version": "2.1.2",
      "resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz",
      "integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==",
-      "dev": true,
+      "license": "MIT",
      "engines": {
        "node": ">=0.8"
      }
@@ -4136,6 +4258,15 @@
        "node": ">=6"
      }
    },
+    "node_modules/cluster-key-slot": {
+      "version": "1.1.2",
+      "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
+      "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
+      "license": "Apache-2.0",
+      "engines": {
+        "node": ">=0.10.0"
+      }
+    },
    "node_modules/codec-parser": {
      "version": "2.5.0",
      "resolved": "https://registry.npmjs.org/codec-parser/-/codec-parser-2.5.0.tgz",
@@ -4531,6 +4662,15 @@
        "node": ">=0.4.0"
      }
    },
+    "node_modules/denque": {
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
+      "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
+      "license": "Apache-2.0",
+      "engines": {
+        "node": ">=0.10"
+      }
+    },
    "node_modules/depd": {
      "version": "2.0.0",
      "resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -4658,6 +4798,20 @@
        "node": ">=10"
      }
    },
+    "node_modules/dunder-proto": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
+      "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
+      "license": "MIT",
+      "dependencies": {
+        "call-bind-apply-helpers": "^1.0.1",
+        "es-errors": "^1.3.0",
+        "gopd": "^1.2.0"
+      },
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
    "node_modules/duplexer3": {
      "version": "0.1.5",
      "resolved": "https://registry.npmjs.org/duplexer3/-/duplexer3-0.1.5.tgz",
@@ -4708,12 +4862,10 @@
      }
    },
    "node_modules/es-define-property": {
-      "version": "1.0.0",
-      "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.0.tgz",
-      "integrity": "sha512-jxayLKShrEqqzJ0eumQbVhTYQM27CfT1T35+gCgDFoL82JLsXqTJ76zv6A0YLOgEnLUMvLzsDsGIrl8NFpT2gQ==",
-      "dependencies": {
-        "get-intrinsic": "^1.2.4"
-      },
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
+      "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
+      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      }
@@ -4746,6 +4898,33 @@
        "url": "https://github.com/sponsors/ljharb"
      }
    },
+    "node_modules/es-object-atoms": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz",
+      "integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==",
+      "license": "MIT",
+      "dependencies": {
+        "es-errors": "^1.3.0"
+      },
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
+    "node_modules/es-set-tostringtag": {
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
+      "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
+      "license": "MIT",
+      "dependencies": {
+        "es-errors": "^1.3.0",
+        "get-intrinsic": "^1.2.6",
+        "has-tostringtag": "^1.0.2",
+        "hasown": "^2.0.2"
+      },
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
    "node_modules/es6-error": {
      "version": "4.1.1",
      "resolved": "https://registry.npmjs.org/es6-error/-/es6-error-4.1.1.tgz",
@@ -5272,6 +5451,31 @@
        "node": "^10.12.0 || >=12.0.0"
      }
    },
+    "node_modules/file-entry-cache/node_modules/flat-cache": {
+      "version": "3.2.0",
+      "resolved": "https://registry.npmjs.org/flat-cache/-/flat-cache-3.2.0.tgz",
+      "integrity": "sha512-CYcENa+FtcUKLmhhqyctpclsq7QF38pKjZHsGNiSQF5r4FtoKDWabFDl3hzaEQMvT1LHEysw5twgLvpYYb4vbw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "flatted": "^3.2.9",
+        "keyv": "^4.5.3",
+        "rimraf": "^3.0.2"
+      },
+      "engines": {
+        "node": "^10.12.0 || >=12.0.0"
+      }
+    },
+    "node_modules/file-entry-cache/node_modules/keyv": {
+      "version": "4.5.4",
+      "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz",
+      "integrity": "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "json-buffer": "3.0.1"
+      }
+    },
    "node_modules/file-type": {
      "version": "16.5.4",
      "resolved": "https://registry.npmjs.org/file-type/-/file-type-16.5.4.tgz",
@@ -5389,24 +5593,21 @@
      }
    },
    "node_modules/flat-cache": {
-      "version": "3.2.0",
-      "resolved": "https://registry.npmjs.org/flat-cache/-/flat-cache-3.2.0.tgz",
-      "integrity": "sha512-CYcENa+FtcUKLmhhqyctpclsq7QF38pKjZHsGNiSQF5r4FtoKDWabFDl3hzaEQMvT1LHEysw5twgLvpYYb4vbw==",
-      "dev": true,
+      "version": "6.1.13",
+      "resolved": "https://registry.npmjs.org/flat-cache/-/flat-cache-6.1.13.tgz",
+      "integrity": "sha512-gmtS2PaUjSPa4zjObEIn4WWliKyZzYljgxODBfxugpK6q6HU9ClXzgCJ+nlcPKY9Bt090ypTOLIFWkV0jbKFjw==",
+      "license": "MIT",
      "dependencies": {
-        "flatted": "^3.2.9",
-        "keyv": "^4.5.3",
-        "rimraf": "^3.0.2"
-      },
-      "engines": {
-        "node": "^10.12.0 || >=12.0.0"
+        "cacheable": "^1.10.4",
+        "flatted": "^3.3.3",
+        "hookified": "^1.11.0"
      }
    },
    "node_modules/flatted": {
-      "version": "3.3.1",
-      "resolved": "https://registry.npmjs.org/flatted/-/flatted-3.3.1.tgz",
-      "integrity": "sha512-X8cqMLLie7KsNUDSdzeN8FYK9rEt4Dt67OsG/DNGnYTSDBG4uFAJFBnUeiV+zCVAvwFy56IjM9sH51jVaEhNxw==",
-      "dev": true
+      "version": "3.3.3",
+      "resolved": "https://registry.npmjs.org/flatted/-/flatted-3.3.3.tgz",
+      "integrity": "sha512-GX+ysw4PBCz0PzosHDepZGANEuFCMLrnRTiEy9McGjmkCQYwRq4A/X786G/fjM/+OjsWSU1ZrY5qyARZmO/uwg==",
+      "license": "ISC"
    },
    "node_modules/follow-redirects": {
      "version": "1.15.9",
@@ -5452,12 +5653,15 @@
      }
    },
    "node_modules/form-data": {
-      "version": "4.0.1",
-      "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.1.tgz",
-      "integrity": "sha512-tzN8e4TX8+kkxGPK8D5u0FNmjPUjw3lwC9lSLxxoB/+GtsJG91CO8bSWy73APlgAZzZbXEYZJuxjkHH2w+Ezhw==",
+      "version": "4.0.4",
+      "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.4.tgz",
+      "integrity": "sha512-KrGhL9Q4zjj0kiUt5OO4Mr/A/jlI2jDYs5eHBpYHPcBEVSiipAvn2Ko2HnPe20rmcuuvMHNdZFp+4IlGTMF0Ow==",
+      "license": "MIT",
      "dependencies": {
        "asynckit": "^0.4.0",
        "combined-stream": "^1.0.8",
+        "es-set-tostringtag": "^2.1.0",
+        "hasown": "^2.0.2",
        "mime-types": "^2.1.12"
      },
      "engines": {
@@ -5642,15 +5846,21 @@
      }
    },
    "node_modules/get-intrinsic": {
-      "version": "1.2.4",
-      "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.2.4.tgz",
-      "integrity": "sha512-5uYhsJH8VJBTv7oslg4BznJYhDoRI6waYCxMmCdnTrcCrHA/fCFKoTFz2JKKE0HdDFUF7/oQuhzumXJK7paBRQ==",
+      "version": "1.3.0",
+      "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
+      "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
+      "license": "MIT",
      "dependencies": {
+        "call-bind-apply-helpers": "^1.0.2",
+        "es-define-property": "^1.0.1",
        "es-errors": "^1.3.0",
+        "es-object-atoms": "^1.1.1",
        "function-bind": "^1.1.2",
-        "has-proto": "^1.0.1",
-        "has-symbols": "^1.0.3",
-        "hasown": "^2.0.0"
+        "get-proto": "^1.0.1",
+        "gopd": "^1.2.0",
+        "has-symbols": "^1.1.0",
+        "hasown": "^2.0.2",
+        "math-intrinsics": "^1.1.0"
      },
      "engines": {
        "node": ">= 0.4"
@@ -5669,6 +5879,19 @@
        "node": ">=8.0.0"
      }
    },
+    "node_modules/get-proto": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
+      "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
+      "license": "MIT",
+      "dependencies": {
+        "dunder-proto": "^1.0.1",
+        "es-object-atoms": "^1.0.0"
+      },
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
    "node_modules/get-stream": {
      "version": "5.2.0",
      "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz",
@@ -5809,11 +6032,12 @@
      }
    },
    "node_modules/gopd": {
-      "version": "1.0.1",
-      "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.0.1.tgz",
-      "integrity": "sha512-d65bNlIadxvpb/A2abVdlqKqV563juRnZ1Wtk6s1sIR8uNsXR70xqIzVqxVf1eTqDunwT2MkczEeaezCKTZhwA==",
-      "dependencies": {
-        "get-intrinsic": "^1.1.3"
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
+      "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.4"
      },
      "funding": {
        "url": "https://github.com/sponsors/ljharb"
@@ -5915,21 +6139,11 @@
        "url": "https://github.com/sponsors/ljharb"
      }
    },
-    "node_modules/has-proto": {
-      "version": "1.0.3",
-      "resolved": "https://registry.npmjs.org/has-proto/-/has-proto-1.0.3.tgz",
-      "integrity": "sha512-SJ1amZAJUiZS+PhsVLf5tGydlaVB8EdFpaSO4gmiUKUOxk8qzn5AIy4ZeJUmh22znIdk/uMAUT2pl3FxzVUH+Q==",
-      "engines": {
-        "node": ">= 0.4"
-      },
-      "funding": {
-        "url": "https://github.com/sponsors/ljharb"
-      }
-    },
    "node_modules/has-symbols": {
-      "version": "1.0.3",
-      "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.0.3.tgz",
-      "integrity": "sha512-l3LCuF6MgDNwTDKkdYGEihYjt5pRPbEg46rtlmnSPlUbgmB8LOIrKJbYYFBSbnPaJexMKtiPO8hmeRjRz2Td+A==",
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
+      "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
+      "license": "MIT",
      "engines": {
        "node": ">= 0.4"
      },
@@ -5941,7 +6155,6 @@
      "version": "1.0.2",
      "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
      "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
-      "dev": true,
      "dependencies": {
        "has-symbols": "^1.0.3"
      },
@@ -6031,6 +6244,12 @@
      "resolved": "https://registry.npmjs.org/react-is/-/react-is-16.13.1.tgz",
      "integrity": "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="
    },
+    "node_modules/hookified": {
+      "version": "1.12.0",
+      "resolved": "https://registry.npmjs.org/hookified/-/hookified-1.12.0.tgz",
+      "integrity": "sha512-hMr1Y9TCLshScrBbV2QxJ9BROddxZ12MX9KsCtuGGy/3SmmN5H1PllKerrVlSotur9dlE8hmUKAOSa3WDzsZmQ==",
+      "license": "MIT"
+    },
    "node_modules/html-escaper": {
      "version": "2.0.2",
      "resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-2.0.2.tgz",
@@ -6243,6 +6462,26 @@
        "node": ">= 0.4"
      }
    },
+    "node_modules/iovalkey": {
+      "version": "0.3.3",
+      "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.3.3.tgz",
+      "integrity": "sha512-4rTJX6Q5wTYEvxboXi8DsEiUo+OvqJGtLYOSGm37KpdRXsG5XJjbVtYKGJpPSWP+QT7rWscA4vsrdmzbEbenpw==",
+      "license": "MIT",
+      "dependencies": {
+        "@iovalkey/commands": "^0.1.0",
+        "cluster-key-slot": "^1.1.0",
+        "debug": "^4.3.4",
+        "denque": "^2.1.0",
+        "lodash.defaults": "^4.2.0",
+        "lodash.isarguments": "^3.1.0",
+        "redis-errors": "^1.2.0",
+        "redis-parser": "^3.0.0",
+        "standard-as-callback": "^2.1.0"
+      },
+      "engines": {
+        "node": ">=18.12.0"
+      }
+    },
    "node_modules/ipaddr.js": {
      "version": "1.9.1",
      "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -6416,6 +6655,19 @@
        "url": "https://github.com/sponsors/ljharb"
      }
    },
+    "node_modules/is-network-error": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/is-network-error/-/is-network-error-1.1.0.tgz",
+      "integrity": "sha512-tUdRRAnhT+OtCZR/LxZelH/C7QtjtFrTu5tXCA8pl55eTUElUHT+GPYV8MBMBvea/j+NxQqVt3LbWMRir7Gx9g==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": ">=16"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
    "node_modules/is-node-process": {
      "version": "1.2.0",
      "resolved": "https://registry.npmjs.org/is-node-process/-/is-node-process-1.2.0.tgz",
@@ -6790,9 +7042,10 @@
      }
    },
    "node_modules/iti": {
-      "version": "0.6.0",
-      "resolved": "https://registry.npmjs.org/iti/-/iti-0.6.0.tgz",
-      "integrity": "sha512-JqujcnAIF3pmzitjbT3acc0LkordU6oHBDvWeT6a25wvEVBddFX3DFx/p6YBwGX1TTFsyLgVZtwhGOknthC96A==",
+      "version": "0.7.0",
+      "resolved": "https://registry.npmjs.org/iti/-/iti-0.7.0.tgz",
+      "integrity": "sha512-ERAkSZdhgrYCKF7zqX7PEGjWpjvotP858ACtpRhXssOo+vdYPD97UUn+/Tzif7oKiabdefQ4NzDETXGHZhZ7dA==",
+      "license": "MIT",
      "dependencies": {
        "utility-types": "^3.10.0"
      },
@@ -6972,7 +7225,8 @@
    "node_modules/json-buffer": {
      "version": "3.0.1",
      "resolved": "https://registry.npmjs.org/json-buffer/-/json-buffer-3.0.1.tgz",
-      "integrity": "sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ=="
+      "integrity": "sha512-4bV5BfR2mqfQTJm+V5tPPdf+ZpuhiIvTuAB5g8kcrXOZpTT/QwwVRWBywX1ozr6lEuPdbHxwaJlm9G6mI2sfSQ==",
+      "license": "MIT"
    },
    "node_modules/json-schema-traverse": {
      "version": "1.0.0",
@@ -7088,11 +7342,12 @@
      }
    },
    "node_modules/keyv": {
-      "version": "4.5.4",
-      "resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz",
-      "integrity": "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw==",
+      "version": "5.5.0",
+      "resolved": "https://registry.npmjs.org/keyv/-/keyv-5.5.0.tgz",
+      "integrity": "sha512-QG7qR2tijh1ftOvClut4YKKg1iW6cx3GZsKoGyJPxHkGWK9oJhG9P3j5deP0QQOGDowBMVQFaP+Vm4NpGYvmIQ==",
+      "license": "MIT",
      "dependencies": {
-        "json-buffer": "3.0.1"
+        "@keyv/serialize": "^1.1.0"
      }
    },
    "node_modules/klaw-sync": {
@@ -7176,6 +7431,12 @@
      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
      "integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg=="
    },
+    "node_modules/lodash.defaults": {
+      "version": "4.2.0",
+      "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
+      "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
+      "license": "MIT"
+    },
    "node_modules/lodash.flattendeep": {
      "version": "4.4.0",
      "resolved": "https://registry.npmjs.org/lodash.flattendeep/-/lodash.flattendeep-4.4.0.tgz",
@@ -7183,6 +7444,12 @@
      "dev": true,
      "peer": true
    },
+    "node_modules/lodash.isarguments": {
+      "version": "3.1.0",
+      "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
+      "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
+      "license": "MIT"
+    },
    "node_modules/lodash.merge": {
      "version": "4.6.2",
      "resolved": "https://registry.npmjs.org/lodash.merge/-/lodash.merge-4.6.2.tgz",
@@ -7272,6 +7539,15 @@
        "url": "https://github.com/sponsors/sindresorhus"
      }
    },
+    "node_modules/math-intrinsics": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
+      "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 0.4"
+      }
+    },
    "node_modules/media-typer": {
      "version": "0.3.0",
      "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz",
@@ -7883,26 +8159,17 @@
      }
    },
    "node_modules/ntfy": {
-      "version": "1.6.2",
-      "resolved": "https://registry.npmjs.org/ntfy/-/ntfy-1.6.2.tgz",
-      "integrity": "sha512-56ITDEDeSYDWWOsZbv4J9qawaJUIHEy0Rga1+FZh6F5eq/17UPMzZ64PhcmuM2YunM1zCTWH2tM5zIELel5S7Q==",
+      "version": "1.7.6",
+      "resolved": "https://registry.npmjs.org/ntfy/-/ntfy-1.7.6.tgz",
+      "integrity": "sha512-CXYWnD2/2FcgqxZzrRw8coBXELJA2rjBf49iZoG2Dh3X6e21bDjjfS/ZkvRihrTULnQA/1Cv6pocUTIEUvqIpg==",
+      "license": "GPL-3.0",
      "dependencies": {
-        "axios": "1.7.4"
+        "axios": "1.11.0"
      },
      "engines": {
        "node": ">= 18.0"
      }
    },
-    "node_modules/ntfy/node_modules/axios": {
-      "version": "1.7.4",
-      "resolved": "https://registry.npmjs.org/axios/-/axios-1.7.4.tgz",
-      "integrity": "sha512-DukmaFRnY6AzAALSH4J2M3k6PkaC+MfaAGdEERRWcC9q3/TWQwLpHR8ZRLKTdQ3aBDL64EdluRDjJqKw+BPZEw==",
-      "dependencies": {
-        "follow-redirects": "^1.15.6",
-        "form-data": "^4.0.0",
-        "proxy-from-env": "^1.1.0"
-      }
-    },
    "node_modules/nyc": {
      "version": "17.1.0",
      "resolved": "https://registry.npmjs.org/nyc/-/nyc-17.1.0.tgz",
@@ -8368,6 +8635,24 @@
        "node": ">=8"
      }
    },
+    "node_modules/p-retry": {
+      "version": "6.2.1",
+      "resolved": "https://registry.npmjs.org/p-retry/-/p-retry-6.2.1.tgz",
+      "integrity": "sha512-hEt02O4hUct5wtwg4H4KcWgDdm+l1bOaEy/hWzd8xtXB9BqxTWBBhb+2ImAtH4Cv4rPjV76xN3Zumqk3k3AhhQ==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@types/retry": "0.12.2",
+        "is-network-error": "^1.0.0",
+        "retry": "^0.13.1"
+      },
+      "engines": {
+        "node": ">=16.17"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
    "node_modules/p-timeout": {
      "version": "3.2.0",
      "resolved": "https://registry.npmjs.org/p-timeout/-/p-timeout-3.2.0.tgz",
@@ -9179,7 +9464,6 @@
      "version": "2.3.1",
      "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz",
      "integrity": "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==",
-      "dev": true,
      "engines": {
        "node": ">=6"
      }
@@ -9462,6 +9746,27 @@
        "node": ">=8"
      }
    },
+    "node_modules/redis-errors": {
+      "version": "1.2.0",
+      "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
+      "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=4"
+      }
+    },
+    "node_modules/redis-parser": {
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
+      "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
+      "license": "MIT",
+      "dependencies": {
+        "redis-errors": "^1.0.0"
+      },
+      "engines": {
+        "node": ">=4"
+      }
+    },
    "node_modules/redux": {
      "version": "4.2.1",
      "resolved": "https://registry.npmjs.org/redux/-/redux-4.2.1.tgz",
@@ -10127,6 +10432,12 @@
      "dev": true,
      "peer": true
    },
+    "node_modules/standard-as-callback": {
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
+      "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
+      "license": "MIT"
+    },
    "node_modules/statuses": {
      "version": "2.0.1",
      "resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.1.tgz",
@@ -10629,27 +10940,6 @@
        "node": ">=0.6.0"
      }
    },
-    "node_modules/tmp-promise": {
-      "version": "3.0.3",
-      "resolved": "https://registry.npmjs.org/tmp-promise/-/tmp-promise-3.0.3.tgz",
-      "integrity": "sha512-RwM7MoPojPxsOBYnyd2hy0bxtIlVrihNs9pj5SUvY8Zz1sQcQG2tG1hSr8PDxfgEB8RNKDhqbIlroIarSNDNsQ==",
-      "dev": true,
-      "dependencies": {
-        "tmp": "^0.2.0"
-      }
-    },
-    "node_modules/tmp-promise/node_modules/tmp": {
-      "version": "0.2.1",
-      "resolved": "https://registry.npmjs.org/tmp/-/tmp-0.2.1.tgz",
-      "integrity": "sha512-76SUhtfqR2Ijn+xllcI5P1oyannHNHByD80W1q447gU3mp9G9PSpGdWmjUOHRDPiHYacIk66W7ubDTuPF3BEtQ==",
-      "dev": true,
-      "dependencies": {
-        "rimraf": "^3.0.0"
-      },
-      "engines": {
-        "node": ">=8.17.0"
-      }
-    },
    "node_modules/to-readable-stream": {
      "version": "2.1.0",
      "resolved": "https://registry.npmjs.org/to-readable-stream/-/to-readable-stream-2.1.0.tgz",
@@ -10992,6 +11282,63 @@
      "resolved": "https://registry.npmjs.org/typescript-event-target/-/typescript-event-target-1.1.1.tgz",
      "integrity": "sha512-dFSOFBKV6uwaloBCCUhxlD3Pr/P1a/tJdcmPrTXCHlEFD3faj0mztjcGn6VBAhQ0/Bdy8K3VWrrqwbt/ffsYsg=="
    },
+    "node_modules/typeson": {
+      "version": "9.0.4",
+      "resolved": "https://registry.npmjs.org/typeson/-/typeson-9.0.4.tgz",
+      "integrity": "sha512-umRYLe37m4fTu0AlgFaqu1C+N+i3LqAehCzbvNx9w7alyN1wpPyS6FbBEsvWD2FppS2yJ9gk0FkFBkZ/wJE13Q==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=16.0.0"
+      }
+    },
+    "node_modules/typeson-registry": {
+      "version": "11.1.1",
+      "resolved": "https://registry.npmjs.org/typeson-registry/-/typeson-registry-11.1.1.tgz",
+      "integrity": "sha512-WF8meNtURfGXXY2RitEMgZrMEclHU8IdxxN4YTcnXTQUHvisPYsKrxxPFhc4DKTCAc6sVvWVtKelOXFDtw/OQQ==",
+      "license": "MIT",
+      "dependencies": {
+        "base64-arraybuffer-es6": "^3.1.0",
+        "typeson": "^9.0.3",
+        "whatwg-url": "^14.0.0"
+      },
+      "engines": {
+        "node": ">=18.14.0"
+      }
+    },
+    "node_modules/typeson-registry/node_modules/tr46": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/tr46/-/tr46-5.1.1.tgz",
+      "integrity": "sha512-hdF5ZgjTqgAntKkklYw0R03MG2x/bSzTtkxmIRw/sTNV8YXsCJ1tfLAX23lhxhHJlEf3CRCOCGGWw3vI3GaSPw==",
+      "license": "MIT",
+      "dependencies": {
+        "punycode": "^2.3.1"
+      },
+      "engines": {
+        "node": ">=18"
+      }
+    },
+    "node_modules/typeson-registry/node_modules/webidl-conversions": {
+      "version": "7.0.0",
+      "resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-7.0.0.tgz",
+      "integrity": "sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g==",
+      "license": "BSD-2-Clause",
+      "engines": {
+        "node": ">=12"
+      }
+    },
+    "node_modules/typeson-registry/node_modules/whatwg-url": {
+      "version": "14.2.0",
+      "resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-14.2.0.tgz",
+      "integrity": "sha512-De72GdQZzNTUBBChsXueQUnPKDkg/5A5zp7pFDuQAj5UFoENpiACU0wlCvzpAGnTkj++ihpKwKyYewn/XNUbKw==",
+      "license": "MIT",
+      "dependencies": {
+        "tr46": "^5.1.0",
+        "webidl-conversions": "^7.0.0"
+      },
+      "engines": {
+        "node": ">=18"
+      }
+    },
    "node_modules/uid-safe": {
      "version": "2.1.5",
      "resolved": "https://registry.npmjs.org/uid-safe/-/uid-safe-2.1.5.tgz",
@@ -11721,16 +12068,18 @@
      }
    },
    "node_modules/with-local-tmp-dir": {
-      "version": "5.1.1",
-      "resolved": "https://registry.npmjs.org/with-local-tmp-dir/-/with-local-tmp-dir-5.1.1.tgz",
-      "integrity": "sha512-7HZx3lC2BNMqrkob/29+GxBZtt0J2j5qyYn77/FivEAKS2Q90Tlc6KaqjBAPnVo0B3OfHDtCBZG3i8S6kD8N8A==",
+      "version": "6.0.0",
+      "resolved": "https://registry.npmjs.org/with-local-tmp-dir/-/with-local-tmp-dir-6.0.0.tgz",
+      "integrity": "sha512-UqVBYzOaK6gXybyO3tYGrATBfQ9itasWQVpO5DPi7/YitEMS0u7wqgJHLpD9CUYSHyIgeKmLw/HN/iIm3dGxCA==",
      "dev": true,
+      "license": "MIT",
      "dependencies": {
-        "@dword-design/chdir": "^2.0.0",
-        "tmp-promise": "^3.0.0"
+        "@dword-design/chdir": "^4.0.0",
+        "@types/fs-extra": "^11.0.4",
+        "p-retry": "^6.2.1"
      },
      "engines": {
-        "node": ">=14"
+        "node": ">=20"
      },
      "funding": {
        "url": "https://github.com/sponsors/dword-design"
@@ -51,12 +51,13 @@
    "@fortawesome/react-fontawesome": "^0.2.0",
    "@foxxmd/chromecast-client": "^1.0.4",
    "@foxxmd/get-version": "^0.0.3",
-    "@foxxmd/logging": "^0.2.2",
+    "@foxxmd/logging": "^0.2.3",
+    "@foxxmd/redact-string": "^0.1.2",
    "@foxxmd/regex-buddy-core": "^0.1.2",
    "@foxxmd/string-sameness": "^0.4.0",
-    "@gr2m/net-interceptor": "^1.0.0",
    "@jellyfin/sdk": "^0.11.0",
    "@kenyip/backoff-strategies": "^1.0.4",
+    "@keyv/valkey": "^1.0.8",
    "@lukehagar/plexjs": "^0.39.0",
    "@react-nano/use-event-source": "^0.13.0",
    "@reduxjs/toolkit": "^1.9.5",
@@ -68,7 +69,9 @@
    "avahi-browse": "^1.1.4",
    "better-sse": "^0.8.0",
    "body-parser": "^1.19.0",
+    "cacheable": "^1.10.4",
    "castv2": "^0.1.10",
+    "clone": "^2.1.2",
    "clsx": "^2.0.0",
    "common-tags": "^1.8.2",
    "compare-versions": "^4.1.2",
@@ -81,6 +84,7 @@
    "express-session": "^1.17.2",
    "fast-deep-equal": "^3.1.3",
    "fixed-size-list": "^0.3.0",
+    "flat-cache": "^6.1.13",
    "formidable": "^3.5",
    "glob": "^11.0.0",
    "google-auth-library": "^9.15.0",
@@ -88,15 +92,16 @@
    "http-cookie-agent": "^7.0.1",
    "icecast-metadata-stats": "^0.1.12",
    "iso-websocket": "^0.3.0",
-    "iti": "^0.6.0",
+    "iti": "^0.7.0",
    "json5": "^2.2.3",
+    "keyv": "^5.5.0",
    "kodi-api": "^0.2.1",
    "lastfm-node-client": "^2.2.0",
    "mopidy": "^1.3.0",
    "mpd-api": "^1.1.2",
    "nanoid": "^3.3.1",
    "normalize-url": "^8.0.1",
-    "ntfy": "^1.6.2",
+    "ntfy": "^1.7.6",
    "object-hash": "^3.0.0",
    "p-event": "^4.2.0",
    "passport": "^0.6.0",
@@ -115,6 +120,8 @@
    "tough-cookie": "^5.1.2",
    "ts-json-schema-generator": "^2.3.0",
    "tsx": "^4.7.0",
+    "typeson": "^9.0.4",
+    "typeson-registry": "^11.1.1",
    "vite-express": "^0.16.0",
    "vlc-client": "^1.1.1",
    "xml2js": "0.6.1",
@@ -124,6 +131,7 @@
    "@dbus-types/notifications": "^0.0.5",
    "@eslint/js": "^8.56.0",
    "@faker-js/faker": "^9.0.1",
+    "@gr2m/net-interceptor": "^1.0.0",
    "@istanbuljs/nyc-config-typescript": "^1.0.2",
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
@@ -153,7 +161,6 @@
    "@vitejs/plugin-react": "^4.2.1",
    "chai": "^4.3.6",
    "chai-as-promised": "^7.1.1",
-    "clone": "^2.1.2",
    "eslint": "^8.56.0",
    "eslint-plugin-prefer-arrow-functions": "^3.2.4",
    "mocha": "^10.3.0",
@@ -164,7 +171,7 @@
    "typescript": "5.5.4",
    "typescript-eslint": "^7.0.1",
    "vite": "^5.2.12",
-    "with-local-tmp-dir": "^5.1.1"
+    "with-local-tmp-dir": "^6.0.0"
  },
  "browserslist": {
    "production": [
@@ -181,11 +188,6 @@
  "overrides": {
    "spotify-web-api-node": {
      "superagent": "$superagent"
-    },
-    "with-local-tmp-dir": {
-      "tmp-promise": {
-        "tmp": "0.2.1"
-      }
    }
  }
 }
@@ -24,7 +24,7 @@ import { CommonClientConfig } from "./infrastructure/config/client/index.js";
 import { CommonSourceConfig } from "./infrastructure/config/source/index.js";
 import play = Simulate.play;
 import { WebhookPayload } from "./infrastructure/config/health/webhooks.js";
-import { AuthCheckError, BuildDataError, ConnectionCheckError, PostInitError, TransformRulesError } from "./errors/MSErrors.js";
+import { AuthCheckError, BuildDataError, ConnectionCheckError, ParseCacheError, PostInitError, TransformRulesError } from "./errors/MSErrors.js";
 import { messageWithCauses, messageWithCausesTruncatedDefault } from "../utils/ErrorUtils.js";

 export default abstract class AbstractComponent {
@@ -35,12 +35,13 @@ export default abstract class AbstractComponent {

    buildOK?: boolean | null;
    connectionOK?: boolean | null;
+    cacheOK?: boolean | null;

    initializing: boolean = false;

    config: CommonClientConfig | CommonSourceConfig;

-    transformRules!: PlayTransformRules;
+    transformRules: PlayTransformRules = {};
    regexCache!: ReturnType<typeof cacheFunctions>;

    logger: Logger;
@@ -65,6 +66,7 @@ export default abstract class AbstractComponent {
                await this.buildComponentLogger();
            }
            await this.buildInitData(force);
+            await this.parseCache(force);
            this.buildTransformRules();
            await this.checkConnection(force);
            await this.testAuth(force);
@@ -105,6 +107,45 @@ export default abstract class AbstractComponent {
        }
    }

+    public async parseCache(force: boolean = false) {
+        if(this.cacheOK) {
+            if(!force) {
+                return;
+            }
+            this.logger.debug('Cache OK but step was forced');
+        }
+        try {
+            const res = await this.doParseCache();
+            if(res === undefined) {
+                this.cacheOK = null;
+                this.logger.debug('No cache to parse.');
+                return;
+            }
+            if (res === true) {
+                this.logger.verbose('Parsing caching succeeded');
+            } else if (typeof res === 'string') {
+                this.logger.verbose(`Parsing caching succeeded => ${res}`);
+            }
+            this.cacheOK = true;
+        } catch (e) {
+            this.cacheOK = false;
+            throw new ParseCacheError('Parsing cache for initialization failed', {cause: e});
+        }
+    }
+
+    /**
+     * Build or parse any cache required for this Component
+     *
+     * * Return undefined if not possible or not required
+     * * Return TRUE if build succeeded
+     * * Return string if build succeeded and should log result
+     * * Throw error on failure
+     * */
+    protected async doParseCache(): Promise<true | string | undefined> {
+        return;
+    }
+
+
    public async buildInitData(force: boolean = false) {
        if(this.buildOK) {
            if(!force) {
@@ -166,7 +207,6 @@ export default abstract class AbstractComponent {
        } = this.config;

        if (playTransform === undefined) {
-            this.transformRules = {};
            return;
        }

@@ -0,0 +1,280 @@
+import { Cacheable, createKeyv, Keyv, KeyvStoreAdapter, KeyvOptions, CacheableOptions } from 'cacheable';
+import { FlatCache, FlatCacheOptions } from 'flat-cache';
+import KeyvValkey from '@keyv/valkey';
+import dayjs, { Dayjs } from 'dayjs';
+import duration from 'dayjs/plugin/duration.js';
+import isBetween from 'dayjs/plugin/isBetween.js';
+import relativeTime from 'dayjs/plugin/relativeTime.js';
+import isToday from 'dayjs/plugin/isToday.js';
+import timezone from 'dayjs/plugin/timezone.js';
+import utc from 'dayjs/plugin/utc.js';
+import clone from 'clone';
+import { childLogger, Logger } from '@foxxmd/logging';
+import { projectDir } from './index.js';
+import path from 'path';
+import { fileOrDirectoryIsWriteable } from '../utils.js';
+import { asCacheMetadataProvider, asCacheScrobbleProvider, CacheConfig, CacheConfigOptions, CacheMetadaProvider, CacheProvider, CacheScrobbleProvider } from './infrastructure/Atomic.js';
+import { Typeson } from 'typeson';
+import { builtin } from 'typeson-registry';
+import { MaybeLogger } from './logging.js';
+import { ListenProgressPositional, ListenProgressTS } from '../sources/PlayerState/ListenProgress.js';
+const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
+
+dayjs.extend(utc)
+dayjs.extend(isBetween);
+dayjs.extend(relativeTime);
+dayjs.extend(duration);
+dayjs.extend(timezone);
+dayjs.extend(isToday);
+
+const typeson = new Typeson().register([
+    builtin,
+]);
+typeson.register({
+    Dayjs: [
+        (x) => dayjs.isDayjs(x),
+        (d: Dayjs) => d.toJSON(),
+        (date) => dayjs(date)
+    ],
+    ListenProgressTS,
+    ListenProgressPositional
+});
+
+export class MSCache {
+
+    config: Required<CacheConfigOptions>
+
+    cacheMetadata: Cacheable;
+    cacheScrobble: Cacheable;
+
+    logger: Logger;
+
+    constructor(logger: Logger, config: CacheConfigOptions = {}) {
+        this.logger = childLogger(logger, 'Cache');
+
+        const {
+            metadata: {
+                provider: mProvider = (process.env.CACHE_METADATA as (CacheMetadaProvider | undefined) ?? 'memory'),
+                connection: mConn = process.env.CACHE_METADATA_CONN,
+                ...restMetadata
+            } = {},
+            scrobble: {
+                provider: sProvider = (process.env.CACHE_SCROBBLE as (CacheScrobbleProvider | undefined) ?? 'file'),
+                connection = (process.env.CACHE_SCROBBLE_CONN ?? configDir),
+                ...restScrobble
+            } = {},
+        } = config;
+
+        this.config = {
+            metadata: {
+                provider: mProvider,
+                connection: mConn,
+                ...restMetadata,
+            },
+            scrobble: {
+                provider: sProvider,
+                connection,
+                ...restScrobble
+            }
+        };
+    }
+
+    init = async () => {
+        // disabled for now
+        //await this.initMetadataCache();
+        await this.initScrobbleCache();
+    }
+
+    protected initCacheable = async (config: CacheConfig, cacheFor: string) => {
+
+        let logger = childLogger(this.logger, cacheFor);
+        const providerHints = ['In-Memory (Primary)'];
+        if(config.provider !== false) {
+            providerHints.push(`${config.provider} (Secondary)`)
+        }
+        logger.verbose(`Cache Providers: ${providerHints.join(' | ')}`)
+
+        const ns = `ms-${cacheFor.toLocaleLowerCase()}`;
+
+        const cacheOpts: CacheableOptions = {
+            primary: initMemoryCache({ namespace: ns })
+        }
+
+        let secondaryCache: Keyv | KeyvStoreAdapter | undefined;
+
+        if (config.provider === 'valkey') {
+            logger.debug(`Building valkey cache from ${config.connection}`);
+            try {
+                secondaryCache = await initValkeyCache(ns, config.connection);
+                logger.debug('valkey cache connected');
+            } catch (e) {
+                this.logger.warn(e);
+            }
+        } else if (config.provider === 'file') {
+            logger.debug(`Building file cache from ${path.join(config.connection, `${ns}.cache`)}`);
+
+            try {
+                const [keyvFile] = await initFileCache({ ...config, cacheDir: config.connection, cacheId: `${ns}.cache` }, logger);
+                secondaryCache = keyvFile;
+            } catch (e) {
+                logger.warn(e);
+            }
+        }
+
+        if(secondaryCache !== undefined) {
+            cacheOpts.secondary = secondaryCache;
+        }
+        return new Cacheable(cacheOpts);
+
+    }
+
+    initScrobbleCache = async () => {
+        if (this.cacheScrobble === undefined) {
+            if (!asCacheScrobbleProvider(this.config.scrobble.provider)) {
+                throw new Error(`Cache Scrobble provider '${this.config.scrobble.provider}' must be one of: memory, valkey, file`);
+            }
+
+            this.cacheScrobble = await this.initCacheable(this.config.scrobble, 'Scrobble');
+        }
+    }
+
+    initMetadataCache = async () => {
+        if (this.cacheMetadata === undefined) {
+            if (!asCacheMetadataProvider(this.config.metadata.provider)) {
+                throw new Error(`Cache Metadata provider '${this.config.metadata.provider}' must be one of: memory, valkey`);
+            }
+
+            this.cacheMetadata = await this.initCacheable(this.config.metadata, 'Metadata');
+        }
+    }
+}
+
+
+export const initMemoryCache = (opts: Parameters<typeof createKeyv>[0] = {}): Keyv | KeyvStoreAdapter => {
+    const memory = createKeyv({
+        ttl: '1h',
+        lruSize: 200,
+        ...opts,
+        useClone: false,
+    });
+    // structuredClone does not work well with dayjs https://github.com/iamkun/dayjs/issues/2236
+    // but deep cloning is fine so disable useClone and provide our own cloning function
+    memory.serialize = (data) => {
+        return clone(data) as string;
+    }
+    return memory;
+}
+
+export const flatCacheCreate = (opts: FlatCacheOptions) => {
+    return new FlatCache({
+        ttl: 0,
+        lruSize: 2000,
+        cacheDir: opts.cacheDir ?? configDir,
+        cacheId: opts.cacheId ?? 'scrobble.cache',
+        persistInterval: 1 * 1000 * 10,
+        expirationInterval: 1 * 1000 * 10, // 10 seconds
+        ...opts
+    });
+}
+
+export const flatCacheLoad = async (flatCache: FlatCache, logger: MaybeLogger): Promise<void> => {
+
+    const cachePath = path.join(flatCache.cacheDir, flatCache.cacheId);
+    try {
+        fileOrDirectoryIsWriteable(cachePath);
+    } catch (e) {
+        throw new Error(`Unable to use path for file cache at ${cachePath}`, { cause: e })
+    }
+
+    const streamPromise = new Promise((resolve, reject) => {
+        flatCache.loadFileStream(cachePath, (progress: number, total: number) => {
+            logger.debug(`Loading ${progress}/${total} chunks...`);
+        }, () => {
+            resolve(true);
+        }, (err: Error) => {
+            reject(err);
+        });
+    });
+
+    try {
+        await streamPromise;
+        logger.debug(`File cache loaded`);
+        return;
+    } catch (e) {
+        if (null !== e.message.match(/Cache file .+ does not exist/)) {
+            let loadError: Error;
+            try {
+                const onlySaveError = (e: Error) => {
+                    loadError = e;
+                };
+                flatCache.on('error', onlySaveError);
+                flatCache.load();
+                if (loadError !== undefined) {
+                    throw loadError;
+                }
+                flatCache.off('error', onlySaveError);
+                logger.debug(`File cache loaded`);
+                return;
+            } catch (e) {
+                throw new Error(`Unable to use file cache at ${cachePath}`, { cause: e });
+            }
+        } else {
+            throw new Error(`Unable to use file cache at ${cachePath}`, { cause: e });
+        }
+    }
+}
+
+export const initFileCache = async (opts: FlatCacheOptions = {}, logger: MaybeLogger = new MaybeLogger()): Promise<[Keyv | KeyvStoreAdapter | undefined, FlatCache | undefined]> => {
+    const flatCache = flatCacheCreate(opts);
+    try {
+        await flatCacheLoad(flatCache, logger);
+        flatCache.on('error', (e) => {
+            logger.warn(e);
+        });
+        flatCache.on('save', () => {
+            logger.debug('Saved cache to file');
+        });
+
+        const cache = new Keyv({
+            store: flatCache,
+            throwOnErrors: true,
+            ...typesonMarshalling
+        });
+        return [cache, flatCache];
+    } catch (e) {
+        throw e;
+    }
+}
+
+export const valkeyCacheCreate = (ns: string, ...args: ConstructorParameters<typeof KeyvValkey>): Keyv => {
+    const [connection, valkeyOpts = {}] = args;
+    const valkey = new KeyvValkey(connection, { maxRetriesPerRequest: 5, connectTimeout: 1100, ...valkeyOpts });
+    const kv = new Keyv({
+        store: valkey,
+        throwOnErrors: true,
+        namespace: ns,
+        ...typesonMarshalling
+    });
+    return kv;
+}
+
+export const initValkeyCache = async (ns: string, ...args: ConstructorParameters<typeof KeyvValkey>): Promise<Keyv> => {
+    const kv = valkeyCacheCreate(ns, ...args);
+    try {
+        await kv.get('test');
+        return kv;
+    } catch (e) {
+        throw new Error(`Unable to connect to cache ${args[0]}`, { cause: e })
+    }
+}
+
+const typesonMarshalling: Pick<KeyvOptions, 'serialize' | 'deserialize'> = {
+    serialize: (data) => {
+        const str = typeson.stringifySync(data);
+        return str;
+    },
+    deserialize: (str) => {
+        const data = typeson.parseSync(str);
+        return data;
+    }
+}
@@ -8,12 +8,16 @@ export class BuildDataError extends StageError {
    name = 'Init Build Data';
 }

+export class ParseCacheError extends StageError {
+    name = 'Init Parse Cache';
+}
+
 export class TransformRulesError extends StageError {
    name = 'Transform Rules';
 }

 export class ConnectionCheckError extends StageError {
-    name = 'Conenction Check';
+    name = 'Connection Check';
 }

 export class AuthCheckError extends StageError {
@@ -345,3 +345,30 @@ export type WhenParts<T> = PlayTransformPartsAtomic<T>;

 export type WhenConditions<T> = WhenParts<T>[];
 export type WhenConditionsConfig = WhenConditions<string>;
+
+export type WithRequiredProperty<Type, Key extends keyof Type> = Type & {
+  [Property in Key]-?: Type[Property];
+};
+export type CacheProvider = 'memory' | 'valkey' | 'file' | false;
+export interface CacheConfig<T extends CacheProvider = CacheProvider> {
+    provider: T;
+    connection?: string;
+    [key: string]: any
+}
+export type CacheMetadaProvider = CacheProvider;//Exclude<CacheProvider, 'file'>;
+export type CacheMetadataConfig = CacheConfig<CacheMetadaProvider>;
+export const asCacheProvider = (val: boolean | string): val is CacheProvider => {
+    if(typeof val === 'string') {
+        return ['memory', 'valkey', 'file'].includes(val);
+    }
+    return val === false;
+}
+export const asCacheMetadataProvider = (val: any): val is CacheScrobbleProvider => asCacheProvider(val);
+export type CacheScrobbleProvider = CacheProvider;
+export type CacheScrobbleConfig = CacheConfig<CacheScrobbleProvider>;
+export const asCacheScrobbleProvider = (val: any): val is CacheScrobbleProvider => asCacheProvider(val);
+export interface CacheConfigOptions {
+    metadata?: CacheMetadataConfig;
+    scrobble?: CacheScrobbleConfig;
+}
+
@@ -5,7 +5,7 @@ import { RequestRetryOptions } from "./common.js";
 import { WebhookConfig } from "./health/webhooks.js";
 import { CommonSourceOptions, SourceRetryOptions } from "./source/index.js";
 import { SourceAIOConfig } from "./source/sources.js";
-import { ClientType, SourceType } from "../Atomic.js";
+import { CacheConfigOptions, ClientType, SourceType } from "../Atomic.js";


 export interface SourceDefaults extends CommonSourceOptions {
@@ -64,6 +64,8 @@ export interface AIOConfig {
     * @examples [false]
     * */
    debugMode?: boolean
+
+    cache?: CacheConfigOptions
 }

 export interface AIOClientConfig {
@@ -82,6 +82,11 @@ export interface NtfyConfig extends CommonWebhookConfig {
     * */
    password?: string

+    /**
+     * Use instead of username/password, required if topic is protected
+     */
+    token?: string
+
    /**
     * Priority of messages
     *
@@ -1,7 +1,7 @@
 import { stringSameness } from '@foxxmd/string-sameness';
 import dayjs from "dayjs";
 import request, { Request, Response } from 'superagent';
-import { BrainzMeta, PlayObject, URLData } from "../../../core/Atomic.js";
+import { BrainzMeta, PlayObject, SCROBBLE_TS_SOC_START, URLData } from "../../../core/Atomic.js";
 import { combinePartsToString, slice } from "../../../core/StringUtils.js";
 import {
    findDelimiters,
@@ -14,11 +14,11 @@ import {
 } from "../../utils/StringUtils.js";
 import { getScrobbleTsSOCDate } from "../../utils/TimeUtils.js";
 import { UpstreamError } from "../errors/UpstreamError.js";
-import { AbstractApiOptions, DEFAULT_RETRY_MULTIPLIER, FormatPlayObjectOptions } from "../infrastructure/Atomic.js";
+import { AbstractApiOptions, DEFAULT_RETRY_MULTIPLIER, DELIMITERS, FormatPlayObjectOptions } from "../infrastructure/Atomic.js";
 import { ListenBrainzClientData } from "../infrastructure/config/client/listenbrainz.js";
 import AbstractApiClient from "./AbstractApiClient.js";
 import { getBaseFromUrl, isPortReachableConnect, joinedUrl, normalizeWebAddress } from '../../utils/NetworkUtils.js';
-import { removeUndefinedKeys } from '../../utils.js';
+import { removeUndefinedKeys, unique } from '../../utils.js';
 import {ListensResponse as KoitoListensResponse} from '../infrastructure/config/client/koito.js'
 import { listenObjectResponseToPlay } from './koito/KoitoApiClient.js';
 import { version } from '../../ioc.js';
@@ -259,17 +259,32 @@ export class ListenbrainzApiClient extends AbstractApiClient {
                    duration,
                    track_mbid,
                    artist_mbids,
+                    artist_names = [],
                    release_mbid,
-                    release_group_mbid
+                    release_group_mbid,
+                    release_artist_name,
+                    release_artist_names = [],
+                    scrobble_ts_soc = SCROBBLE_TS_SOC_START,
+                    listened_for,
+                    listened_at_completed
                } = {}
            } = {},
        } = payload;

-        return {
+        let albumArtists: string[];
+        if(release_artist_name !== undefined) {
+            albumArtists = [release_artist_name];
+        }
+        if(release_artist_names.length > 0) {
+            albumArtists = unique([...(albumArtists ?? []), ...release_artist_names])
+        }
+
+        const play: PlayObject = {
            data: {
                playDate: typeof listened_at === 'number' ? dayjs.unix(listened_at) : dayjs(listened_at),
                track: track_name,
-                artists: [artist_name],
+                artists: unique([artist_name, ...artist_names]),
+                albumArtists,
                album: release_name,
                duration,
                meta: {
@@ -283,8 +298,18 @@ export class ListenbrainzApiClient extends AbstractApiClient {
            },
            meta: {
                nowPlaying,
+                scrobbleTsSOC: scrobble_ts_soc
            }
        }
+
+        if(listened_at_completed !== undefined) {
+            play.data.playDateCompleted = dayjs.unix(listened_at_completed);
+        }
+        if(listened_for !== undefined) {
+            play.data.listenedFor = listened_for;
+        }
+
+        return play;
    }

    static listenResponseToPlay(listen: ListenResponse): PlayObject {
@@ -395,7 +420,15 @@ export class ListenbrainzApiClient extends AbstractApiClient {
            }

            // now try to extract any remaining artists from filtered artist/name values
-            const parsedArtists = parseArtistCredits(filteredSubmittedArtistName);
+            const splitAmpersand = artistsWithJoiners.length === 0 && artistMappings.some(x => x.join_phrase.includes('&'));
+            let nonProperJoinedDelims = undefined;
+            if(artistsWithJoiners.length === 0) {
+                nonProperJoinedDelims = unique(artistMappings.filter(x => DELIMITERS.includes(x.join_phrase.trim())).map(x => x.join_phrase.trim()));
+                if(nonProperJoinedDelims.length === 0) {
+                    nonProperJoinedDelims = undefined;
+                }
+            }
+            const parsedArtists = parseArtistCredits(filteredSubmittedArtistName, nonProperJoinedDelims);
            if (parsedArtists !== undefined) {
                if (parsedArtists.primary !== undefined) {
                    artistsFromUserValues.push(parsedArtists.primary);
@@ -3,6 +3,8 @@
 * https://musicbrainz.org/doc/MusicBrainz_Database/Schema#Overview
 */

+import { ScrobbleTsSOC } from "../../../../core/Atomic.js";
+
 /** A unique product a Recording is issued on.
 *
 * This is like an album (release group) but is specific to the type, year, catalog, etc... for this release
@@ -123,6 +125,21 @@ export interface SubmitListenAdditionalTrackInfo extends AdditionalTrackInfo {
    spotify_album_artist_ids?: string[];
    spotify_artist_ids?: string[];
    albumartist?: string;
+    /** Is the timestamp (listened_at) from when the user started listening or when they stopped listening?
+     * 
+     * Specific to multi-scrobbler
+     */
+    scrobble_ts_soc?: ScrobbleTsSOC
+    /** Unix timestamp for when the user stopped listening
+     * 
+     * Specific to multi-scrobbler
+     */
+    listened_at_completed?: number
+    /** Number of seconds track was listened to
+     * 
+     * Specific to multi-scrobbler
+     */
+    listened_for?: number
 }
 export interface TrackPayload extends MinimumTrack {
    additional_info?: SubmitListenAdditionalTrackInfo;
@@ -20,6 +20,7 @@ import { isDebugMode, parseBool, readJson, retry, sleep } from "./utils.js";
 import { createVegaGenerator } from './utils/SchemaUtils.js';
 import ScrobbleClients from './scrobblers/ScrobbleClients.js';
 import ScrobbleSources from './sources/ScrobbleSources.js';
+import { Notifiers } from './notifier/Notifiers.js';

 dayjs.extend(utc)
 dayjs.extend(isBetween);
@@ -95,7 +96,16 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
        createVegaGenerator()
        initLogger.info('Schema definitions generated');

-        initServer(logger, appLoggerStream, output);
+        const scrobbleClients = new ScrobbleClients(root.get('clientEmitter'), root.get('sourceEmitter'), root.get('localUrl'), root.get('configDir'), root.get('logger'));
+        const scrobbleSources = new ScrobbleSources(root.get('sourceEmitter'), {
+             localUrl: root.get('localUrl'),
+            configDir: root.get('configDir'),
+             version: root.get('version')
+             }, root.get('logger'));
+
+        await root.items.cache().init();
+
+        initServer(logger, appLoggerStream, output, scrobbleSources, scrobbleClients);

        if(process.env.IS_LOCAL === 'true') {
            logger.info('multi-scrobbler can be run as a background service! See: https://foxxmd.github.io/multi-scrobbler/docs/installation/service');
@@ -106,18 +116,16 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
            logger.warn(appConfigFail);
        }

-        const notifiers = root.get('notifiers');
+        const notifiers = new Notifiers(root.get('notifierEmitter'), root.get('clientEmitter'), root.get('sourceEmitter'), root.get('logger')); //root.get('notifiers');
        await notifiers.buildWebhooks(webhooks);

        /*
        * setup clients
        * */
-        const scrobbleClients = root.get('clients') as ScrobbleClients;
        await scrobbleClients.buildClientsFromConfig(notifiers);
        /*
        * setup sources
        * */
-        const scrobbleSources = root.get('sources') as ScrobbleSources;
        await scrobbleSources.buildSourcesFromConfig([]);

        // check ambiguous client/source types like this for now
@@ -1,16 +1,15 @@
 import { getVersion } from "@foxxmd/get-version";
-import { Logger, LogOptions } from "@foxxmd/logging";
+import { Logger, loggerDebug, LogOptions } from "@foxxmd/logging";
 import { EventEmitter } from "events";
 import { createContainer } from "iti";
 import path from "path";
 import { projectDir } from "./common/index.js";
 import { WildcardEmitter } from "./common/WildcardEmitter.js";
-import { Notifiers } from "./notifier/Notifiers.js";
-import ScrobbleClients from "./scrobblers/ScrobbleClients.js";
-import ScrobbleSources from "./sources/ScrobbleSources.js";

 import { generateBaseURL } from "./utils/NetworkUtils.js";
 import { PassThrough } from "stream";
+import { CacheConfigOptions } from "./common/infrastructure/Atomic.js";
+import { MSCache } from "./common/Cache.js";

 export let version: string = 'unknown';

@@ -22,20 +21,23 @@ let root: ReturnType<typeof createRoot>;

 export interface RootOptions {
    baseUrl?: string,
-    port?: string | number
+    port?: number
    logger: Logger
    disableWeb?: boolean
    loggerStream?: PassThrough
    loggingConfig?: LogOptions
+    cache?: CacheConfigOptions | MSCache | (() => MSCache)
 }

-const createRoot = (options?: RootOptions) => {
+const createRoot = (options: RootOptions = {logger: loggerDebug}) => {
    const {
        port = 9078,
        baseUrl = process.env.BASE_URL,
        disableWeb: dw,
        loggerStream,
-        loggingConfig
+        loggingConfig,
+        logger,
+        cache
    } = options || {};
    const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
    let disableWeb = dw;
@@ -43,6 +45,18 @@ const createRoot = (options?: RootOptions) => {
        disableWeb = process.env.DISABLE_WEB === 'true';
    }

+    let cacheFunc: () => MSCache;
+    let maybeSingletonCache: MSCache;
+
+    if(cache instanceof MSCache) {
+        maybeSingletonCache = cache;
+    } else if(typeof cache === 'function') {
+        cacheFunc = cache;
+    } else {
+        maybeSingletonCache = new MSCache(logger, cache);
+    }
+
+
    const cEmitter = new WildcardEmitter();
    // do nothing, just catch
    cEmitter.on('error', (e) => null);
@@ -51,23 +65,25 @@ const createRoot = (options?: RootOptions) => {
        const f = e;
    });

+    const portVal: number | string = process.env.PORT ?? port;
+
    return createContainer().add({
        version,
        configDir: configDir,
        isProd: process.env.NODE_ENV !== undefined && (process.env.NODE_ENV === 'production' || process.env.NODE_ENV === 'prod'),
-        port: process.env.PORT ?? port,
+        // @ts-ignore
+        port: (Number.isInteger(portVal) ? portVal : Number.parseInt(portVal)) as number,
        disableWeb,
        clientEmitter: () => cEmitter,
        sourceEmitter: () => sEmitter,
        notifierEmitter: () => new EventEmitter(),
        loggerStream,
        loggingConfig,
+        logger: logger,
+        cache: () => maybeSingletonCache !== undefined ? () => maybeSingletonCache : cacheFunc
    }).add((items) => {
        const localUrl = generateBaseURL(baseUrl, items.port)
        return {
-            clients: () => new ScrobbleClients(items.clientEmitter, items.sourceEmitter, localUrl, items.configDir, options.logger),
-            sources: () => new ScrobbleSources(items.sourceEmitter, { localUrl, configDir: items.configDir, version }, options.logger),
-            notifiers: () => new Notifiers(items.notifierEmitter, items.clientEmitter, items.sourceEmitter, options.logger),
            localUrl,
            hasDefinedBaseUrl: baseUrl !== undefined,
            isSubPath: localUrl.pathname !== '/' && localUrl.pathname.length > 0
@@ -1,6 +1,7 @@
 import { Logger } from "@foxxmd/logging";
 import { Config, publish } from 'ntfy';
 import request from "superagent";
+import {redactString} from '@foxxmd/redact-string';
 import { NtfyConfig, PrioritiesConfig, WebhookPayload } from "../common/infrastructure/config/health/webhooks.js";
 import { AbstractWebhookNotifier } from "./AbstractWebhookNotifier.js";
 import { URLData } from "../../core/Atomic.js";
@@ -33,7 +34,14 @@ export class NtfyWebhookNotifier extends AbstractWebhookNotifier {
    initialize = async () => {
        // check url is correct as a courtesy
        this.endpoint = normalizeWebAddress(this.config.url);
-        this.logger.verbose(`Config URL: '${this.config.url}' => Normalized: '${this.endpoint.normal}'`)
+        this.logger.verbose(`Config URL: '${this.config.url}' => Normalized: '${this.endpoint.normal}'`);
+        if(this.config.token !== undefined) {
+            this.logger.verbose(`Using Access Token '${redactString(this.config.token, 3)}' for authentication`);
+        } else if(this.config.username !== undefined) {
+            this.logger.verbose(`Using Username/Password '${redactString(this.config.username, 3)}/${redactString(this.config.password, 3)}' for authentication`);
+        } else {
+            this.logger.verbose('No authentication provided, will not be able to push to protected topics');
+        }

        this.initialized = true; // always set as ready to go. Server issues may be transient.

@@ -72,7 +80,9 @@ export class NtfyWebhookNotifier extends AbstractWebhookNotifier {
                server: this.endpoint.normal,
                priority: this.priorities[payload.priority],
            };
-            if (this.config.username !== undefined) {
+            if(this.config.token !== undefined) {
+                req.authorization = `Bearer ${this.config.token.replace(/Bearer/i, '').trim()}`;
+            } else if (this.config.username !== undefined) {
                req.authorization = {
                    username: this.config.username,
                    password: this.config.password,
@@ -52,6 +52,9 @@ import {
 } from "../utils/TimeUtils.js";
 import { WebhookPayload } from "../common/infrastructure/config/health/webhooks.js";
 import { AsyncTask, SimpleIntervalJob, Task, ToadScheduler } from "toad-scheduler";
+import { MSCache } from "../common/Cache.js";
+import { getRoot } from "../ioc.js";
+import { rehydratePlay } from "../utils/CacheUtils.js";

 type PlatformMappedPlays = Map<string, {play: PlayObject, source: SourceIdentifier}>;
 type NowPlayingQueue = Map<string, PlatformMappedPlays>;
@@ -97,6 +100,8 @@ export default abstract class AbstractScrobbleClient extends AbstractComponent i
    nowPlayingTaskInterval: number = 5000;
    npLogger: Logger;

+    cache: MSCache;
+
    declare config: CommonClientConfig;

    notifier: Notifiers;
@@ -110,6 +115,7 @@ export default abstract class AbstractScrobbleClient extends AbstractComponent i
        this.npLogger = childLogger(this.logger, 'Now Playing');
        this.notifier = notifier;
        this.emitter = emitter;
+        this.cache = getRoot().items.cache();

        this.scrobbledPlayObjs = new FixedSizeList<ScrobbledPlayObject>(this.MAX_STORED_SCROBBLES);

@@ -169,6 +175,9 @@ export default abstract class AbstractScrobbleClient extends AbstractComponent i
    protected getIdentifier() {
        return `${capitalize(this.type)} - ${this.name}`
    }
+    protected getMachineId() {
+        return `${this.type}-${this.name}`;
+    }

    public notify = async (payload: WebhookPayload) => {
        this.emitEvent('notify', payload);
@@ -295,6 +304,18 @@ export default abstract class AbstractScrobbleClient extends AbstractComponent i
        }
    }

+    protected async doParseCache(): Promise<true | string | undefined> {
+        const cachedQueue = (await this.cache.cacheScrobble.get(`${this.getMachineId()}-queue`) as QueuedScrobble<PlayObject>[] ?? []);
+        const cachedQLength = cachedQueue.length;
+        this.queuedScrobbles = cachedQueue.map(x => ({...x, play: rehydratePlay(x.play)}));
+
+        const cachedDead = (await this.cache.cacheScrobble.get(`${this.getMachineId()}-dead`) as DeadLetterScrobble<PlayObject>[] ?? []);
+        const cachedDLength = cachedDead.length;
+        this.deadLetterScrobbles = cachedDead.map(x => ({...x, play: rehydratePlay(x.play), lastRetry: x.lastRetry !== undefined ? dayjs(x.lastRetry) : undefined}));
+
+        return `Scrobbles from Cache: ${cachedQLength} Queue | ${cachedDLength} Dead Letter`;
+    }
+
    protected async postInitialize(): Promise<void> {
        const {
            options: {
@@ -770,8 +791,9 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
                        await this.refreshScrobbles();
                    }
                    const currQueuedPlay = this.queuedScrobbles.shift();
+
                    const [timeFrameValid, timeFrameValidLog] = this.timeFrameIsValid(currQueuedPlay.play);
-                    if (timeFrameValid && !(await this.alreadyScrobbled(currQueuedPlay.play))) {
+                    if (timeFrameValid && !(await this.alreadyScrobbled(this.transformPlay(currQueuedPlay.play, TRANSFORM_HOOK.preCompare)))) {
                        const transformedScrobble = this.transformPlay(currQueuedPlay.play, TRANSFORM_HOOK.postCompare);
                        try {
                            const scrobbledPlay = await this.scrobble(transformedScrobble);
@@ -783,12 +805,14 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
                                this.logger.warn(new Error(`Could not scrobble ${buildTrackString(transformedScrobble)} from Source '${currQueuedPlay.source}' but error was not show stopping. Adding scrobble to Dead Letter Queue and will retry on next heartbeat.`, {cause: e}));
                            } else {
                                this.queuedScrobbles.unshift(currQueuedPlay);
+                                this.updateQueuedScrobblesCache();
                                throw new Error('Error occurred while trying to scrobble', {cause: e});
                            }
                        }
                    } else if (!timeFrameValid) {
                        this.logger.debug(`Will not scrobble ${buildTrackString(currQueuedPlay.play)} from Source '${currQueuedPlay.source}' because it ${timeFrameValidLog}`);
                    }
+                    this.updateQueuedScrobblesCache();
                    this.emitEvent('scrobbleDequeued', {queuedScrobble: currQueuedPlay})
                }
                await sleep(this.scrobbleSleep);
@@ -854,7 +878,7 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
            await this.refreshScrobbles();
        }
        const [timeFrameValid, timeFrameValidLog] = this.timeFrameIsValid(deadScrobble.play);
-        if (timeFrameValid && !(await this.alreadyScrobbled(deadScrobble.play))) {
+        if (timeFrameValid && !(await this.alreadyScrobbled(this.transformPlay(deadScrobble.play, TRANSFORM_HOOK.preCompare)))) {
            const transformedScrobble = this.transformPlay(deadScrobble.play, TRANSFORM_HOOK.postCompare);
            try {
                const scrobbledPlay = await this.scrobble(transformedScrobble);
@@ -887,10 +911,12 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
        }
        this.logger.info(`Removed scrobble ${buildTrackString(this.deadLetterScrobbles[index].play)} from queue`, {leaf: 'Dead Letter'});
        this.deadLetterScrobbles.splice(index, 1);
+        this.updateDeadLetterCache();
    }

    removeDeadLetterScrobbles = () => {
        this.deadLetterScrobbles = [];
+        this.updateDeadLetterCache();
        this.logger.info('Removed all scrobbles from queue', {leaf: 'Dead Letter'});
    }

@@ -904,12 +930,12 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
    queueScrobble = (data: PlayObject | PlayObject[], source: string) => {
        const plays = Array.isArray(data) ? data : [data];
        for(const p of plays) {
-            const transformedPlay = this.transformPlay(p, TRANSFORM_HOOK.preCompare);
-            const queuedPlay = {id: nanoid(), source, play: transformedPlay}
+            const queuedPlay = {id: nanoid(), source, play: p}
            this.emitEvent('scrobbleQueued', {queuedPlay: queuedPlay});
            this.queuedScrobbles.push(queuedPlay);
        }
        this.queuedScrobbles.sort((a, b) => sortByOldestPlayDate(a.play, b.play));
+        this.updateQueuedScrobblesCache();
    }

    protected addDeadLetterScrobble = (data: QueuedScrobble<PlayObject>, error: (Error | string) = 'Unspecified error') => {
@@ -923,6 +949,7 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
        this.deadLetterScrobbles.push(deadData);
        this.deadLetterScrobbles.sort((a, b) => sortByOldestPlayDate(a.play, b.play));
        this.emitEvent('deadLetter', {dead: deadData});
+        this.updateDeadLetterCache();
    }

    queuePlayingNow = (data: PlayObject, source: SourceIdentifier) => {
@@ -1002,4 +1029,16 @@ ${closestMatch.breakdowns.join('\n')}`, {leaf: ['Dupe Check']});
            from: 'client'
        });
    }
+
+    protected updateDeadLetterCache = () => {
+        this.cache.cacheScrobble.set(`${this.getMachineId()}-dead`, this.deadLetterScrobbles)
+        .then(() => isDebugMode() ? this.logger.debug('Updated dead letter cache') : null)
+        .catch((e) => this.logger.warn(new Error('Error while updating dead letter cache', {cause: e})));
+    }
+
+    protected updateQueuedScrobblesCache = () => {
+        this.cache.cacheScrobble.set(`${this.getMachineId()}-queue`, this.queuedScrobbles)
+        .then(() => isDebugMode() ? this.logger.debug('Updated queued scrobble cache') : null)
+        .catch((e) => this.logger.warn(new Error('Error while updating queued scrobble cache', {cause: e})));
+    }
 }
@@ -47,16 +47,17 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
    }

    getScrobblesForRefresh = async (limit: number) => {
-            const resp = await this.api.callApi<UserGetRecentTracksResponse>((client: any) => client.userGetRecentTracks({
-                user: this.api.user,
-                sk: this.api.client.sessionKey,
-                limit: limit,
-                extended: true
-            }));
+        const resp = await this.api.callApi<UserGetRecentTracksResponse>((client: any) => client.userGetRecentTracks({
+            user: this.api.user,
+            sk: this.api.client.sessionKey,
+            limit: limit,
+            extended: true
+        }));
+        try {
            const {
                recenttracks: {
                    track: list = [],
-                }
+                } = {}
            } = resp;
            return list.reduce((acc: any, x: any) => {
                try {
@@ -71,24 +72,28 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
                            nowPlaying,
                        }
                    } = formatted;
-                    if(nowPlaying === true) {
+                    if (nowPlaying === true) {
                        // if the track is "now playing" it doesn't get a timestamp so we can't determine when it started playing
                        // and don't want to accidentally count the same track at different timestamps by artificially assigning it 'now' as a timestamp
                        // so we'll just ignore it in the context of recent tracks since really we only want "tracks that have already finished being played" anyway
-                        this.logger.debug("Ignoring 'now playing' track returned from Last.fm client", {track, mbid});
+                        this.logger.debug("Ignoring 'now playing' track returned from Last.fm client", { track, mbid });
                        return acc;
-                    } else if(playDate === undefined) {
-                        this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, {track, mbid});
+                    } else if (playDate === undefined) {
+                        this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, { track, mbid });
                        return acc;
                    }
                    return acc.concat(formatted);
                } catch (e) {
-                    this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', {error: e.message});
+                    this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', { error: e.message });
                    this.logger.debug('Full api response object:');
                    this.logger.debug(x);
                    return acc;
                }
            }, []);
+        } catch (e) {
+            this.logger.debug(resp);
+            throw e;
+        }
    }

    cleanSourceSearchTitle = (playObj: PlayObject) => {
@@ -12,6 +12,7 @@ import {
    LogOutputConfig,
    PlayObject,
    SOURCE_SOT,
+    SOURCE_SOT_TYPES,
    SourceStatusData,
 } from "../../core/Atomic.js";
 import { capitalize } from "../../core/StringUtils.js";
@@ -30,6 +31,8 @@ import { makeClientCheckMiddle, makeSourceCheckMiddle } from "./middleware.js";
 import { setupPlexRoutes } from "./plexRoutes.js";
 import { setupTautulliRoutes } from "./tautulliRoutes.js";
 import { setupWebscrobblerRoutes } from "./webscrobblerRoutes.js";
+import ScrobbleSources from "../sources/ScrobbleSources.js";
+import ScrobbleClients from "../scrobblers/ScrobbleClients.js";

 const maxBufferSize = 300;
 const output: Record<number, FixedSizeList<LogDataPretty>> =  {};
@@ -51,7 +54,7 @@ const getLogs = (minLevel: number, limit: number = maxBufferSize, sort: 'asc' |
    return allLogs.flat(1).sort((a, b) => a.time - b.time).slice(0, limit);
 }

-export const setupApi = (app: ExpressWithAsync, logger: Logger, appLoggerStream: PassThrough, initialLogOutput: LogDataPretty[] = []) => {
+export const setupApi = (app: ExpressWithAsync, logger: Logger, appLoggerStream: PassThrough, initialLogOutput: LogDataPretty[] = [], scrobbleSources: ScrobbleSources, scrobbleClients: ScrobbleClients) => {
    for(const level of Object.keys(logger.levels.labels)) {
        output[level] = new FixedSizeList<LeveledLogData>(maxBufferSize);
    }
@@ -90,9 +93,6 @@ export const setupApi = (app: ExpressWithAsync, logger: Logger, appLoggerStream:
        }
    });

-    const scrobbleSources = root.get('sources');
-    const scrobbleClients = root.get('clients');
-
    const clientMiddleFunc = makeClientCheckMiddle(scrobbleClients);
    const sourceMiddleFunc = makeSourceCheckMiddle(scrobbleSources);

@@ -169,9 +169,7 @@ export const setupApi = (app: ExpressWithAsync, logger: Logger, appLoggerStream:

    app.getAsync('/api/status', async (req, res, next) => {

-        const ss = root.get('sources');
-
-        const sourceData = ss.sources.map((x) => {
+        const sourceData = scrobbleSources.sources.map((x) => {
            const {
                type,
                tracksDiscovered = 0,
@@ -193,7 +191,7 @@ export const setupApi = (app: ExpressWithAsync, logger: Logger, appLoggerStream:
                hasAuthInteraction: requiresAuthInteraction,
                authed,
                players: 'players' in x ? (x as MemorySource).playersToObject() : {},
-                sot: ('playerSourceOfTruth' in x) ? x.playerSourceOfTruth : SOURCE_SOT.HISTORY,
+                sot: ('playerSourceOfTruth' in x) ? x.playerSourceOfTruth as SOURCE_SOT_TYPES : SOURCE_SOT.HISTORY,
                supportsUpstreamRecentlyPlayed: x.supportsUpstreamRecentlyPlayed,
                supportsManualListening: x.supportsManualListening,
                manualListening: x.manualListening,
@@ -13,11 +13,13 @@ import { getRoot } from "../ioc.js";
 import { parseBool } from "../utils.js";
 import { getAddress } from "../utils/NetworkUtils.js";
 import { setupApi } from "./api.js";
+import ScrobbleSources from '../sources/ScrobbleSources.js';
+import ScrobbleClients from '../scrobblers/ScrobbleClients.js';

 const app = addAsync(express());
 const router = Router();

-export const initServer = async (parentLogger: Logger, appLoggerStream: PassThrough, initialOutput: LogDataPretty[] = []) => {
+export const initServer = async (parentLogger: Logger, appLoggerStream: PassThrough, initialOutput: LogDataPretty[] = [], sources: ScrobbleSources, clients: ScrobbleClients) => {

    const logger = childLogger(parentLogger, 'API'); // parentLogger.child({labels: ['API']}, mergeArr);

@@ -48,7 +50,7 @@ export const initServer = async (parentLogger: Logger, appLoggerStream: PassThro
        const local = root.get('localUrl');
        const localDefined = root.get('hasDefinedBaseUrl');

-        setupApi(app, logger, appLoggerStream, initialOutput);
+        setupApi(app, logger, appLoggerStream, initialOutput, sources, clients);

        const addy = getAddress();
        const addresses: string[] = [];
@@ -78,7 +80,7 @@ export const initServer = async (parentLogger: Logger, appLoggerStream: PassThro
        app.use('/docs', express.static(path.resolve(projectDir, `./docsite/build`)));

        if(process.env.USE_HASH_ROUTER === undefined) {
-            process.env.USE_HASH_ROUTER = root.get('isSubPath');
+            process.env.USE_HASH_ROUTER = root.get('isSubPath').toString();
        }

        ViteExpress.config({
@@ -308,21 +308,34 @@ export default class JellyfinApiSource extends MemoryPositionalSource {


        if(session.NowPlayingItem !== undefined) {
+            const {
+                Path
+            } = session.NowPlayingItem;
+
            const allowedLibraries = this.getAllowedLibraries();
-            if(allowedLibraries.length > 0 && !allowedLibraries.map(x => x.paths).flat(1).some(x => session.NowPlayingItem.Path.includes(x))) {
+            if(allowedLibraries.length > 0 && (Path === undefined || !allowedLibraries.map(x => x.paths).flat(1).some(x => Path.includes(x)))) {
+                if(Path === undefined) {
+                    return 'media does not have a path, cannot be included in librariesAllow';
+                }
                return `media not included in librariesAllow`;
            }
            
            if(allowedLibraries.length === 0) {
                const blockedLibraries = this.getBlockedLibraries();
                if(blockedLibraries.length > 0) {
-                    const blockedLibrary = blockedLibraries.find(x => x.paths.some(y => session.NowPlayingItem.Path.includes(y)));
+                    let blockedLibrary = undefined;
+                    if(Path !== undefined) {
+                        blockedLibrary = blockedLibraries.find(x => x.paths.some(y => Path.includes(y)));
+                    }
                    if(blockedLibrary !== undefined) {
                        return `media included in librariesBlock '${blockedLibrary.name}'`;
                    }
                }
    
-                if(!this.getValidLibraries().map(x => x.paths).flat(1).some(x => session.NowPlayingItem.Path.includes(x))) {
+                if(Path === undefined || !this.getValidLibraries().map(x => x.paths).flat(1).some(x => Path.includes(x))) {
+                    if(Path === undefined) {
+                        return 'media does not have a path, cannot be part of a valid library';
+                    }
                    return `media not included in a valid library`;
                }
            }
@@ -80,47 +80,52 @@ export default class LastfmSource extends MemorySource {
            limit,
            extended: true
        }));
-        const {
-            recenttracks: {
-                track: list = [],
-            }
-        } = resp;
+        try {
+            const {
+                recenttracks: {
+                    track: list = [],
+                } = {}
+            } = resp;

-        const plays = list.reduce((acc: PlayObject[], x: TrackObject) => {
-            try {
-                const formatted = LastfmApiClient.formatPlayObj(x);
-                const {
-                    data: {
-                        track,
-                        playDate,
-                    },
-                    meta: {
-                        mbid,
-                        nowPlaying,
+            const plays = list.reduce((acc: PlayObject[], x: TrackObject) => {
+                try {
+                    const formatted = LastfmApiClient.formatPlayObj(x);
+                    const {
+                        data: {
+                            track,
+                            playDate,
+                        },
+                        meta: {
+                            mbid,
+                            nowPlaying,
+                        }
+                    } = formatted;
+                    if (playDate === undefined) {
+                        if (nowPlaying === true) {
+                            formatted.data.playDate = dayjs();
+                            return acc.concat(formatted);
+                        }
+                        this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, { track, mbid });
+                        return acc;
                    }
-                } = formatted;
-                if(playDate === undefined) {
-                    if(nowPlaying === true) {
-                        formatted.data.playDate = dayjs();
-                        return acc.concat(formatted);
-                    }
-                    this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, {track, mbid});
+                    return acc.concat(formatted);
+                } catch (e) {
+                    this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', { error: e.message });
+                    this.logger.debug('Full api response object:');
+                    this.logger.debug(x);
                    return acc;
                }
-                return acc.concat(formatted);
-            } catch (e) {
-                this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', {error: e.message});
-                this.logger.debug('Full api response object:');
-                this.logger.debug(x);
-                return acc;
-            }
-        }, []).sort(sortByOldestPlayDate);
-        // if the track is "now playing" it doesn't get a timestamp so we can't determine when it started playing
-        // and don't want to accidentally count the same track at different timestamps by artificially assigning it 'now' as a timestamp
-        // so we'll just ignore it in the context of recent tracks since really we only want "tracks that have already finished being played" anyway
-        const history = plays.filter(x => x.meta.nowPlaying !== true);
-        const now = plays.filter(x => x.meta.nowPlaying === true);
-        return [history, now];
+            }, []).sort(sortByOldestPlayDate);
+            // if the track is "now playing" it doesn't get a timestamp so we can't determine when it started playing
+            // and don't want to accidentally count the same track at different timestamps by artificially assigning it 'now' as a timestamp
+            // so we'll just ignore it in the context of recent tracks since really we only want "tracks that have already finished being played" anyway
+            const history = plays.filter(x => x.meta.nowPlaying !== true);
+            const now = plays.filter(x => x.meta.nowPlaying === true);
+            return [history, now];
+        } catch (e) {
+            this.logger.debug(resp);
+            throw e;
+        }
    }

    getRecentlyPlayed = async(options: RecentlyPlayedOptions = {}): Promise<PlayObject[]> => {
@@ -97,7 +97,7 @@ const getGoogleOauthOpts = (): GenerateAuthUrlOpts => {
            return x;
        }).flat(1);
    } else {
-        scopes = VALID_SCOPES;
+        scopes = DEFAULT_SCOPES;
    }

    return {
@@ -448,6 +448,10 @@ Redirect URI  : ${this.redirectUri}`);
            });
            return res;
        } catch (e) {
+            const info = loggedErrorExtra(e);
+            if(info !== undefined) {
+                this.logger.error(info, 'Additional API response details')
+            }
            throw e;
        }
    }
@@ -0,0 +1,187 @@
+import { loggerTest, loggerDebug } from "@foxxmd/logging";
+import chai, { assert, expect } from 'chai';
+import asPromised from 'chai-as-promised';
+import { after, before, describe, it } from 'mocha';
+import dayjs from "dayjs";
+import withLocalTmpDir from 'with-local-tmp-dir';
+import path from 'path';
+import { initFileCache, initMemoryCache, initValkeyCache, MSCache } from "../../common/Cache.js";
+import { generatePlay, generatePlayerStateData, generatePlays, normalizePlays } from "../utils/PlayTestUtils.js";
+import { ListenProgressPositional, ListenProgressTS } from "../../sources/PlayerState/ListenProgress.js";
+import { isPortReachableConnect } from "../../utils/NetworkUtils.js";
+import { getRoot } from "../../ioc.js";
+import { transientCache } from "../utils/CacheTestUtils.js";
+import { TestScrobbler } from "../scrobbler/TestScrobbler.js";
+import { sleep } from "../../utils.js";
+import {promises} from 'node:fs';
+
+chai.use(asPromised);
+
+describe('#Caching', function () {
+
+    describe('#MemoryCaching', function () {
+
+        it('Memory cache preserves dayjs', async function () {
+
+            const cache = initMemoryCache();
+
+            const now = dayjs();
+
+            await cache.set('foo', now);
+
+            const time = await cache.get('foo');
+
+            expect(time).to.not.be.undefined;
+            expect(dayjs.isDayjs(time)).is.true;
+            expect(now.toJSON()).eq((time as any).toJSON());
+        });
+
+        it('Memory cache preserves ListenProgress', async function () {
+
+            const cache = initMemoryCache();
+
+            const prog = new ListenProgressPositional({ timestamp: dayjs(), position: 35, positionPercent: 50 });
+
+            await cache.set('foo', prog);
+
+            const cachedProg = await cache.get('foo') as ListenProgressTS;
+
+            expect(cachedProg).to.not.be.undefined;
+            expect(cachedProg instanceof ListenProgressTS).is.true;
+            expect(cachedProg.timestamp.toJSON()).eq(prog.timestamp.toJSON());
+        });
+
+    });
+
+    describe('#FileCaching', function () {
+
+        it('File cache serializes and deserializes dayjs', async function () {
+
+            withLocalTmpDir(async () => {
+
+                const [keyv, flat] = await initFileCache({ cacheDir: process.cwd() });
+
+                const now = dayjs();
+
+                await keyv.set('foo', now);
+                flat.save();
+
+                const [cleanKeyv, cleanFlat] = await initFileCache({ cacheDir: process.cwd() });
+
+                const time = await cleanKeyv.get('foo');
+
+                expect(time).to.not.be.undefined;
+                expect(time instanceof dayjs).is.true;
+                expect(now.toJSON()).eq((time as any).toJSON());
+                flat.destroy();
+
+            }, { unsafeCleanup: true });
+        });
+
+        it('File cache serializes and deserializes ListenProgress', async function () {
+
+            withLocalTmpDir(async () => {
+
+                const [keyv, flat] = await initFileCache({ cacheDir: process.cwd() });
+
+                const prog = new ListenProgressPositional({ timestamp: dayjs(), position: 35, positionPercent: 50 });
+
+                await keyv.set('foo', prog);
+                await flat.save();
+
+                const [cleanKeyv, cleanFlat] = await initFileCache({ cacheDir: process.cwd() });
+
+                const cachedProg = await cleanKeyv.get('foo');
+
+                expect(cachedProg).to.not.be.undefined;
+                expect(cachedProg instanceof ListenProgressTS).is.true;
+                expect(cachedProg.timestamp.toJSON()).eq(prog.timestamp.toJSON());
+                flat.destroy();
+
+            }, { unsafeCleanup: true });
+        });
+    });
+
+    describe('#ValkeyCaching', function () {
+        before(async function () {
+            try {
+                await isPortReachableConnect(6379, { host: 'valkey' });
+            } catch (e) {
+                // don't run valkey tests if valkey isn't present
+                this.skip();
+            }
+        });
+
+        it('Valkey cache serializes and deserializes dayjs', async function () {
+
+            const keyv = await initValkeyCache('test', 'redis://valkey:6379');
+            await keyv.clear();
+
+            const now = dayjs();
+
+            await keyv.set('foo', now);
+
+            const time = await keyv.get('foo');
+
+            expect(time).to.not.be.undefined;
+            expect(time instanceof dayjs).is.true;
+            expect(now.toJSON()).eq((time as any).toJSON());
+
+        });
+
+        it('Valkey cache serializes and deserializes ListenProgress', async function () {
+
+            const keyv = await initValkeyCache('test', 'redis://valkey:6379');
+            await keyv.clear();
+
+            const prog = new ListenProgressPositional({ timestamp: dayjs(), position: 35, positionPercent: 50 });
+
+            await keyv.set('foo', prog);
+
+            const cachedProg = await keyv.get('foo');
+
+            expect(cachedProg).to.not.be.undefined;
+            expect(cachedProg instanceof ListenProgressTS).is.true;
+            expect(cachedProg.timestamp.toJSON()).eq(prog.timestamp.toJSON());
+
+        });
+    });
+
+    describe('#ScrobbleCache', function () {
+
+        afterEach(function () {
+            const root = getRoot();
+            root.upsert({ cache: () => transientCache });
+            root.items.cache().init();
+        });
+
+        it('Preserves scrobbles', async function () {
+
+
+           await withLocalTmpDir(async () => {
+
+                const root = getRoot();
+                root.upsert({ cache: () => () => new MSCache(loggerTest, { scrobble: { provider: 'file', connection: process.cwd(), persistInterval: 100 } }) });
+
+                const test = new TestScrobbler();
+                await test.initialize();
+                const plays = generatePlays(100);
+                test.queueScrobble(plays, 'testSource');
+                const queued = test.queuedScrobbles.map(x => x.play);
+                await sleep(101);
+                const dirContents = await promises.readdir('.');
+                const hasCache = dirContents.some(x => x === 'ms-scrobble.cache');
+                expect(hasCache).is.true;
+
+                const newTest = new TestScrobbler();
+                await newTest.initialize();
+                expect(newTest.queuedScrobbles.length).to.eq(plays.length);
+                expect(newTest.queuedScrobbles[0].play.data.track).to.eq(queued[0].data.track);
+
+            }, { unsafeCleanup: true });
+
+        });
+
+    });
+
+});
@@ -30,7 +30,7 @@ describe('Play Transforms', function () {

    beforeEach(function() {
       component.config = {};
-       component.transformRules = undefined;
+       component.transformRules = {};
    });

    describe('Transform Config Parsing', function() {
@@ -74,7 +74,7 @@
            {
              "artist_credit_name": "Metro Boomin",
              "artist_mbid": "59db3d82-86ea-451f-881f-dffc8ec387c9",
-              "join_phrase": " & "
+              "join_phrase": " , "
            },
            {
              "artist_credit_name": "James Blake",
@@ -116,7 +116,7 @@
            {
              "artist_credit_name": "Childish Gambino",
              "artist_mbid": "7fb57fba-a6ef-44c2-abab-2fa3bdee607e",
-              "join_phrase": " & "
+              "join_phrase": " , "
            },
            {
              "artist_credit_name": "Ariana Grande",
@@ -1,33 +1,34 @@
 import { loggerTest } from "@foxxmd/logging";
-import { assert } from 'chai';
+import { assert, expect } from 'chai';
 import dayjs from "dayjs";
 import { describe, it } from 'mocha';
 import { http, HttpResponse } from "msw";
 import { PlayObject } from "../../../core/Atomic.js";
 import { UpstreamError } from "../../common/errors/UpstreamError.js";

-import { ListenbrainzApiClient } from "../../common/vendor/ListenbrainzApiClient.js";
+import { ListenbrainzApiClient, playToListenPayload } from "../../common/vendor/ListenbrainzApiClient.js";
 import { ListenResponse } from '../../common/vendor/listenbrainz/interfaces.js';
 import { ExpectedResults } from "../utils/interfaces.js";
 import { withRequestInterception } from "../utils/networking.js";
-import artistWithProperJoiner from './correctlyMapped/artistProperHasJoinerInName.json';
+import artistWithProperJoiner from './correctlyMapped/artistProperHasJoinerInName.json' with { type: "json" };
 // correct mappings
-import multiArtistInArtistName from './correctlyMapped/multiArtistInArtistName.json';
-import multiArtistsInTrackName from './correctlyMapped/multiArtistInTrackName.json';
-import multiMappedArtistsWithSingleUserArtist from './correctlyMapped/multiArtistMappingWithSingleRecordedArtist.json';
-import noArtistMapping from './correctlyMapped/noArtistMapping.json';
-import normalizedValues from './correctlyMapped/normalizedName.json';
-import slightlyDifferentNames from './correctlyMapped/trackNameSlightlyDifferent.json';
+import multiArtistInArtistName from './correctlyMapped/multiArtistInArtistName.json' with { type: "json" };
+import multiArtistsInTrackName from './correctlyMapped/multiArtistInTrackName.json' with { type: "json" };
+import multiMappedArtistsWithSingleUserArtist from './correctlyMapped/multiArtistMappingWithSingleRecordedArtist.json' with { type: "json" };
+import noArtistMapping from './correctlyMapped/noArtistMapping.json' with { type: "json" };
+import normalizedValues from './correctlyMapped/normalizedName.json' with { type: "json" };
+import slightlyDifferentNames from './correctlyMapped/trackNameSlightlyDifferent.json' with { type: "json" };

 // incorrect mappings
-import incorrectMultiArtistsTrackName from './incorrectlyMapped/multiArtistsInTrackName.json';
-import veryWrong from './incorrectlyMapped/veryWrong.json';
+import incorrectMultiArtistsTrackName from './incorrectlyMapped/multiArtistsInTrackName.json' with { type: "json" };
+import veryWrong from './incorrectlyMapped/veryWrong.json' with { type: "json" };
+import { generatePlay } from "../utils/PlayTestUtils.js";

 interface LZTestFixture {
    data: ListenResponse
    expected: ExpectedResults
 }
-describe('Listenbrainz Listen Parsing', function () {
+describe('#PlayParse Listenbrainz Listen Parsing', function () {

    describe('When user-submitted artist/track do NOT match MB mappings', function() {
        it('Uses user submitted values when no artist mappings', async function () {
@@ -56,7 +57,7 @@ describe('Listenbrainz Listen Parsing', function () {
    })


-    describe('When user-submitted artist/track matches a MB mapped value', function() {
+    describe('#PlayParse When user-submitted artist/track matches a MB mapped value', function() {

        it('Detects slightly different track names as equal', async function () {
            for(const test of slightlyDifferentNames as unknown as LZTestFixture[]) {
@@ -151,3 +152,37 @@ describe('Listenbrainz Response Behavior', function() {
        }
    ));
 });
+
+describe('Listenbrainz Endpoint Behavior', function() {
+
+    it('Should combine artist and artist_names', function() {
+
+        const play = generatePlay({artists: ['Artist A'], albumArtists: []});
+        const submitPayload = playToListenPayload(play);
+
+        const additionalArtists = [...submitPayload.track_metadata.additional_info.artist_names, 'Artist B'];
+
+        submitPayload.track_metadata.additional_info.artist_names = additionalArtists;
+
+        const playFromPayload = ListenbrainzApiClient.listenPayloadToPlay(submitPayload);
+
+        expect(playFromPayload.data.artists).to.be.eql(additionalArtists)
+        
+    });
+
+    it('Should combine artist and artist_names into a unique array', function() {
+
+        const play = generatePlay({artists: ['Artist A'], albumArtists: []});
+        const submitPayload = playToListenPayload(play);
+
+        const additionalArtists = ['Artist A', 'Artist B'];
+
+        submitPayload.track_metadata.additional_info.artist_names = additionalArtists;
+
+        const playFromPayload = ListenbrainzApiClient.listenPayloadToPlay(submitPayload);
+
+        expect(playFromPayload.data.artists).to.be.eql(['Artist A', 'Artist B'])
+        
+    });
+
+});
@@ -0,0 +1,140 @@
+import { loggerTest, loggerDebug, childLogger } from "@foxxmd/logging";
+import chai, { assert, expect } from 'chai';
+import asPromised from 'chai-as-promised';
+import { after, before, describe, it } from 'mocha';
+
+import { asPlays, generateArtistsStr, generatePlay, normalizePlays } from "../utils/PlayTestUtils.js";
+import { parseArtistCredits, parseContextAwareStringList, parseCredits } from "../../utils/StringUtils.js";
+
+describe('#PlayParse Parsing Artists from String', function() {
+
+    it('Parses Artists from an Artist-like string', function () {
+        for(const i of Array(40)) {
+            const [str, primaries, secondaries] = generateArtistsStr({primary: {max: 3, ambiguousJoinedNames: true, trailingAmpersand: true, finalJoiner: false}});
+            const credits = parseArtistCredits(str);
+            const allArtists = primaries.concat(secondaries);
+            const parsed = [credits.primary].concat(credits.secondary ?? []);
+            expect(primaries.concat(secondaries),`
+'${str}'
+Expected => ${allArtists.join(' || ')}
+Found    => ${parsed.join(' || ')}`)
+
+.eql(parsed)
+        }
+    });
+
+    it('Parses & as "local" joiner when other delimiters present', function () {
+
+        const data = [
+        {
+            str: `Melendi \\ Ryan Lewis \\ The Righteous Brothers (featuring Joan Jett & The Blackhearts \\ Robin Schulz)`,
+            expected: ['Melendi', 'Ryan Lewis', 'The Righteous Brothers', 'Joan Jett & The Blackhearts', 'Robin Schulz']
+        }, 
+        {
+            str: `Gigi D'Agostino \\ YOASOBI (vs Sam Hunt, Lisa Loeb & Booba)`,
+            expected: [`Gigi D'Agostino`, 'YOASOBI', 'Sam Hunt', 'Lisa Loeb', 'Booba']
+        },
+        {
+            str: `Wham!, Hillsong Worship & Bruce Channel feat. I Prevail`,
+            expected: ['Wham!', 'Hillsong Worship & Bruce Channel', 'I Prevail']
+        }
+    ];
+
+        for(const d of data) {
+            const credits = parseArtistCredits(d.str);
+            const parsed = [credits.primary].concat(credits.secondary ?? [])
+            expect(d.expected).eql(parsed)
+        }
+    
+    });
+
+    it('Only parses & as "global" joiner when no other delimiters present', function () {
+
+        const data = [{
+            str: `Melendi & Ryan Lewis & The Righteous Brothers (featuring The Blackhearts \\ Robin Schulz)`,
+            expected: ['Melendi', 'Ryan Lewis', 'The Righteous Brothers', 'The Blackhearts', 'Robin Schulz']
+        }];
+
+        for(const d of data) {
+            const credits = parseArtistCredits(d.str);
+            const parsed = [credits.primary].concat(credits.secondary ?? [])
+            expect(d.expected).eql(parsed)
+        }
+    });
+
+    it('Does not split artist name when only one joiner is present', function () {
+
+        const data = [
+            {
+            str: `Melendi & Ryan Lewis`,
+            expected: ['Melendi & Ryan Lewis']
+        },{
+            str: `Melendi and Ryan Lewis`,
+            expected: ['Melendi and Ryan Lewis']
+        },
+    ];
+
+        for(const d of data) {
+            const credits = parseArtistCredits(d.str);
+            const parsed = [credits.primary].concat(credits.secondary ?? [])
+            expect(d.expected).eql(parsed)
+        }
+    });
+
+    it('Parses secondary free regex', function () {
+
+        const data = [{
+            str: `Diddy & Grand Funk Railroad feat. Daya & (G)I-DLE`,
+            expected: ['Diddy', 'Grand Funk Railroad', 'Daya', '(G)I-DLE']
+        }];
+
+        for(const d of data) {
+            const credits = parseArtistCredits(d.str);
+            const parsed = [credits.primary].concat(credits.secondary ?? [])
+            expect(d.expected).eql(parsed)
+        }
+    });
+
+    it('Parses singlar Artist with wrapped vs multiple', function () {
+        const [str, primaries, secondaries] = generateArtistsStr({primary: 1, secondary: {num: 2, ft: 'vs', joiner: '/', ftWrap: true}});
+        const credits = parseArtistCredits(str);
+        const moreCredits = parseCredits(str);
+        expect(true).eq(true);
+    });
+
+    describe('When joiner is known', function () {
+
+        it('Parses many primary artists', function () {
+            for(const i of Array(10)) {
+                const [str, primaries, secondaries] = generateArtistsStr({primary: {max: 3, joiner: '/'}, secondary: 0});
+                const credits = parseArtistCredits(str, ['/']);
+                const allArtists = primaries.concat(secondaries);
+                const parsed = [credits.primary].concat(credits.secondary ?? [])
+                expect(primaries.concat(secondaries),`
+'${str}'
+Expected => ${allArtists.join(' || ')}
+Found    => ${parsed.join(' || ')}`)
+        .eql(parsed)
+            }
+        });
+
+        it('Parses many secondary artists', function () {
+            // fails on -- Peso Pluma / Lil Baby / R. Kelly (featuring TOMORROW X TOGETHER / AC/DC / DaVido)
+            for(const i of Array(10)) {
+                const [str, primaries, secondaries] = generateArtistsStr({primary: {max: 3, joiner: '/'}, secondary: {joiner: '/', finalJoiner: false}});
+                const credits = parseArtistCredits(str, ['/']);
+                const allArtists = primaries.concat(secondaries);
+                const parsed = [credits.primary].concat(credits.secondary ?? [])
+                expect(primaries.concat(secondaries),`
+'${str}'
+Expected => ${allArtists.join(' || ')}
+Found    => ${parsed.join(' || ')}`)
+        .eql(parsed)
+            }
+        });
+
+
+    });
+
+
+});
@@ -28,6 +28,11 @@ export class TestScrobbler extends AbstractScrobbleClient {
        return (await this.existingScrobble(playObj)) !== undefined;
    }

+    protected async doParseCache() {
+        await this.cache.init();
+        return super.doParseCache();
+    }
+


    playToClientPayload(playObject: PlayObject): object {
@@ -8,8 +8,8 @@ import { http, HttpResponse } from 'msw';
 import pEvent from 'p-event';
 import { PlayObject } from "../../../core/Atomic.js";
 import { genGroupIdStr, sleep } from "../../utils.js";
-import mixedDuration from '../plays/mixedDuration.json';
-import withDuration from '../plays/withDuration.json';
+import mixedDuration from '../plays/mixedDuration.json' with { type: 'json' };
+import withDuration from '../plays/withDuration.json' with { type: 'json' };
 import { MockNetworkError, withRequestInterception } from "../utils/networking.js";
 import { asPlays, generatePlay, generatePlayPlatformId, generatePlays, normalizePlays } from "../utils/PlayTestUtils.js";
 import MockDate from 'mockdate';
@@ -567,23 +567,26 @@ describe('Scrobble client uses transform plays correctly', function() {
        //testScrobbler.initScrobbleMonitoring().catch(console.error);
    });

-    it('Transforms play before queue when preCompare is present', async function() {
-        testScrobbler.config.options = {
-            playTransform: {
-                preCompare: {
-                    title: [
-                        'cool'
-                    ]
-                }
-            }
-        }
-        testScrobbler.buildTransformRules();
-        const newScrobble = generatePlay({
-            track: 'my cool track'
-        });
-        testScrobbler.queueScrobble(newScrobble, 'test');
-        expect(testScrobbler.queuedScrobbles[0].play.data.track).is.eq('my  track');
-    });
+    // TODO need to find a better way to detect this
+    // since we are now doing it in the processing loop instead of before queue
+    
+    // it('Transforms play before queue when preCompare is present', async function() {
+    //     testScrobbler.config.options = {
+    //         playTransform: {
+    //             preCompare: {
+    //                 title: [
+    //                     'cool'
+    //                 ]
+    //             }
+    //         }
+    //     }
+    //     testScrobbler.buildTransformRules();
+    //     const newScrobble = generatePlay({
+    //         track: 'my cool track'
+    //     });
+    //     testScrobbler.queueScrobble(newScrobble, 'test');
+    //     expect(testScrobbler.queuedScrobbles[0].play.data.track).is.eq('my  track');
+    // });

    it('Transforms play on scrobble when postCompare is present', async function() {
        testScrobbler.config.options = {
@@ -661,7 +664,7 @@ describe('Scrobble client uses transform plays correctly', function() {

 describe('Manages scrobble queue', function() {

-    before(async function() {
+    beforeEach(async function() {
        testScrobbler = generateTestScrobbler();
        await testScrobbler.initialize();
        testScrobbler.recentScrobbles = normalizedWithMixedDur;
@@ -0,0 +1,6 @@
+import { loggerTest } from '@foxxmd/logging';
+import { getRoot } from "../ioc.js";
+import { transientCache } from './utils/CacheTestUtils.js';
+
+const root = getRoot({cache: transientCache, logger: loggerTest});
+root.items.cache().init();
@@ -0,0 +1,4 @@
+import { loggerTest } from "@foxxmd/logging";
+import { MSCache } from "../../common/Cache.js";
+
+export const transientCache = () => new MSCache(loggerTest, {scrobble: {provider: 'memory'}});
@@ -5,9 +5,11 @@ import isBetween from "dayjs/plugin/isBetween.js";
 import relativeTime from "dayjs/plugin/relativeTime.js";
 import timezone from "dayjs/plugin/timezone.js";
 import utc from "dayjs/plugin/utc.js";
-import { JsonPlayObject, ObjectPlayData, PlayMeta, PlayObject } from "../../../core/Atomic.js";
+import { FEAT, JOINERS, JOINERS_FINAL, JsonPlayObject, ObjectPlayData, PlayMeta, PlayObject } from "../../../core/Atomic.js";
 import { sortByNewestPlayDate } from "../../utils.js";
 import { NO_DEVICE, NO_USER, PlayerStateDataMaybePlay, PlayPlatformId, ReportedPlayerStatus } from '../../common/infrastructure/Atomic.js';
+import { arrayListAnd } from '../../../core/StringUtils.js';
+import { findDelimiters } from '../../utils/StringUtils.js';

 dayjs.extend(utc)
 dayjs.extend(isBetween);
@@ -176,3 +178,121 @@ export const generatePlayPlatformId = (deviceId?: string, userId?: string): Play
 export const generatePlays = (numberOfPlays: number, data: ObjectPlayData = {}, meta: PlayMeta = {}): PlayObject[] => {
    return Array.from(Array(numberOfPlays), () => generatePlay(data, meta));
 }
+
+export const generateArtist = () => faker.music.artist;
+
+export interface ArtistGenerationOptions
+{
+    ambiguousJoinedNames?: boolean, 
+    trailingAmpersand?: boolean
+}
+
+export const generateArtists = (num?: number, max: number = 3, opts: ArtistGenerationOptions = {}) => {
+    if(num === 0 || max === 0) {
+        return [];
+    }
+    let artists = faker.helpers.multiple(faker.music.artist, {count: {min: num ?? 1, max: num ?? max}});
+
+    const {
+        trailingAmpersand = false,
+        ambiguousJoinedNames = false
+    } = opts;
+
+    if(!trailingAmpersand) {
+        // its really hard to parse an artist name that contains an '&' when it comes at the end of a list
+        // because its ambigious if the list is joining the list with & or if & is part of the artist name
+        // so by default don't generate these (we test for specific scenarios in playParsing.test.ts)
+        while(artists[artists.length - 1].includes('&')) {
+            artists = artists.slice(0, artists.length - 1).concat(faker.music.artist());
+        }
+    }
+    if(!ambiguousJoinedNames) {
+        artists = artists.map(x => {
+            let a = x;
+            let foundDelims = findDelimiters(a);
+            while(foundDelims !== undefined && foundDelims.length > 0 && !(foundDelims.length === 1 && foundDelims[0] === '&')) {
+                a = faker.music.artist();
+                foundDelims = findDelimiters(a);
+            }
+            return a;
+        });
+    }
+    return artists;
+}
+
+export interface ArtistGenerateOptions extends ArtistGenerationOptions {
+    num?: number
+    max?: number
+    joiner?: string
+    finalJoiner?: false | string
+    spacedJoiners?: boolean
+}
+
+export interface SecondaryArtistGenerateOptions extends ArtistGenerateOptions {
+    ft?: string
+    ftWrap?: boolean
+}
+
+export interface CompoundArtistGenerateOptions {
+    primary?: number | ArtistGenerateOptions
+    secondary?: number | SecondaryArtistGenerateOptions
+}
+
+export const generateArtistsStr = (options: CompoundArtistGenerateOptions = {}): [string, string[], string[]] => {
+
+    const {primary = {}, secondary = {}} = options;
+
+    const primaryOpts: ArtistGenerateOptions = typeof primary === 'number' ? {num: primary} : primary;
+    const secondaryOpts: SecondaryArtistGenerateOptions = typeof secondary === 'number' ? {num: secondary} : secondary;
+
+    const primaryArt = generateArtists(primaryOpts.num, primaryOpts.max)
+    const secondaryArt = generateArtists(secondaryOpts.num, secondaryOpts.max);
+
+
+    const joinerPrimary: string = primaryOpts.joiner ?? faker.helpers.arrayElement(JOINERS);
+    let finalJoinerPrimary: string = joinerPrimary;
+    if(primaryOpts.finalJoiner !== false) {
+        if(primaryOpts.finalJoiner === undefined) {
+            if(joinerPrimary === ',' && !primaryArt.some(x => x.includes('&'))) {
+                finalJoinerPrimary = faker.helpers.arrayElement(JOINERS_FINAL);
+            }
+        
+        } else {
+            finalJoinerPrimary = primaryOpts.finalJoiner;
+        }
+    }
+
+    const primaryStr = arrayListAnd(primaryArt, joinerPrimary, finalJoinerPrimary, primaryOpts.spacedJoiners);
+
+    if(secondaryArt.length === 0) {
+        return [primaryStr, primaryArt, []];
+    }
+
+    const joinerSecondary: string = secondaryOpts.joiner ?? faker.helpers.arrayElement(JOINERS);
+    let finalJoinerSecondary: string = joinerSecondary;
+    if(secondaryOpts.finalJoiner !== false) {
+        if(secondaryOpts.finalJoiner === undefined) {
+            if(joinerSecondary === ',' && !secondaryArt.some(x => x.includes('&'))) {
+                finalJoinerSecondary = faker.helpers.arrayElement(JOINERS_FINAL);
+            }
+        } else {
+            finalJoinerSecondary = secondaryOpts.finalJoiner;
+        }
+    }
+
+    const secondaryStr = arrayListAnd(secondaryArt, joinerSecondary, finalJoinerSecondary, secondaryOpts.spacedJoiners);
+    const ft = secondaryOpts.ft ?? faker.helpers.arrayElement(FEAT);
+    let sec = `${ft} ${secondaryStr}`;
+    let wrap: boolean;
+    if(secondaryOpts.ftWrap !== undefined) {
+        wrap = secondaryOpts.ftWrap;
+    } else {
+        wrap = faker.datatype.boolean();
+    }
+    if(wrap) {
+        sec = `(${sec})`;
+    }
+    const artistStr = `${primaryStr} ${sec}`;
+
+    return [artistStr, primaryArt, secondaryArt];
+}
@@ -9,7 +9,7 @@ import {
    uniqueNormalizedStrArr
 } from "../../utils/StringUtils.js";
 import { ExpectedResults } from "./interfaces.js";
-import testData from './playTestData.json';
+import testData from './playTestData.json' with { type: "json" };
 import { splitByFirstFound } from '../../../core/StringUtils.js';

 interface PlayTestFixture {
@@ -0,0 +1,46 @@
+import dayjs from 'dayjs';
+import duration from 'dayjs/plugin/duration.js';
+import isBetween from 'dayjs/plugin/isBetween.js';
+import relativeTime from 'dayjs/plugin/relativeTime.js';
+import isToday from 'dayjs/plugin/isToday.js';
+import timezone from 'dayjs/plugin/timezone.js';
+import utc from 'dayjs/plugin/utc.js';
+import { AmbPlayObject, ListenRangeData, ListenRangeDataAmb, PlayObject, PlayProgress, PlayProgressAmb } from '../../core/Atomic.js';
+import objectHash from 'object-hash';
+
+dayjs.extend(utc)
+dayjs.extend(isBetween);
+dayjs.extend(relativeTime);
+dayjs.extend(duration);
+dayjs.extend(timezone);
+dayjs.extend(isToday);
+
+export const rehydratePlay = (obj: AmbPlayObject): PlayObject => {
+    if(obj.data?.playDate !== undefined && typeof obj.data.playDate === 'string') {
+        obj.data.playDate = dayjs(obj.data.playDate);
+        if(obj.data.playDateCompleted !== undefined) {
+            obj.data.playDateCompleted = dayjs(obj.data.playDateCompleted);
+        }
+    
+        if(obj.data.listenRanges !== undefined) {
+            obj.data.listenRanges = obj.data.listenRanges.map(rehydrateListenRangeData);
+        }
+    }
+    return obj as PlayObject;
+}
+
+// this may become problematic since we aren't re-instantiating Progress class, just implementing interface
+// but that may only be an issue if rehydrating source data which isn't in scope so far
+export const rehydrateListenRangeData = (obj: ListenRangeDataAmb): ListenRangeData => {
+    return {
+        start: rehydratePlayProgress(obj.start),
+        end: rehydratePlayProgress(obj.end)
+    } as ListenRangeData;
+}
+
+export const rehydratePlayProgress = (obj: PlayProgressAmb): PlayProgress => {
+    if(typeof obj.timestamp === 'string') {
+        obj.timestamp = dayjs(obj.timestamp);
+    }
+    return obj as PlayProgress;
+}
@@ -1,7 +1,7 @@
 import { strategies, stringSameness, StringSamenessResult } from "@foxxmd/string-sameness";
 import { PlayObject } from "../../core/Atomic.js";
 import { asPlayerStateData, DELIMITERS, PlayerStateDataMaybePlay } from "../common/infrastructure/Atomic.js";
-import { genGroupIdStr, getPlatformIdFromData, parseRegexSingleOrFail } from "../utils.js";
+import { genGroupIdStr, getPlatformIdFromData, intersect, parseRegexSingleOrFail } from "../utils.js";
 import { buildTrackString } from "../../core/StringUtils.js";

 const {levenStrategy, diceStrategy} = strategies;
@@ -61,7 +61,7 @@ export const SECONDARY_CAPTURED_REGEX = new RegExp(/[([]\s*(?<joiner>ft\.?\W|fea
 *    !!!!  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *******
 *
 * */
-export const SECONDARY_FREE_REGEX = new RegExp(/^\s*(?<joiner>ft\.?\W|feat\.?\W|featuring|vs\.?\W)\s*(?<credits>(?:.+?(?= - |\s*[([]))|(?:.*))(?<creditsSuffix>.*)/i);
+export const SECONDARY_FREE_REGEX = new RegExp(/^\s*(?<joiner>ft\.?\W|feat\.?\W|featuring|vs\.?\W)\s*(?<credits>(?:.+?(?= - |\s*[([].+[)\]]$))|(?:.*))(?<creditsSuffix>.*)/i);

 const SECONDARY_REGEX_STRATS: RegExp[] = [SECONDARY_CAPTURED_REGEX, SECONDARY_FREE_REGEX];

@@ -116,7 +116,7 @@ export const parseCredits = (str: string, delimiters?: boolean | string[]): Play
        for(const strat of SECONDARY_REGEX_STRATS) {
            const secCredits = parseRegexSingleOrFail(strat, results.named.secondary);
            if(secCredits !== undefined) {
-                secondary = parseStringList(secCredits.named.credits as string, delims)
+                secondary = parseContextAwareStringList(secCredits.named.credits as string, delims)
                suffix = secCredits.named.creditsSuffix;
                break;
            }
@@ -134,7 +134,7 @@ export const parseCredits = (str: string, delimiters?: boolean | string[]): Play
    }
    return undefined;
 }
-export const parseArtistCredits = (str: string, delimiters?: boolean | string[]): PlayCredits | undefined => {
+export const parseArtistCredits = (str: string, delimiters?: boolean | string[], ignoreGlobalAmpersand?: boolean): PlayCredits | undefined => {
    if (str.trim() === '') {
        return undefined;
    }
@@ -148,7 +148,7 @@ export const parseArtistCredits = (str: string, delimiters?: boolean | string[])
    if (withJoiner !== undefined) {
        // all this does is make sure and "ft" or parenthesis/brackets are separated --
        // it doesn't also separate primary artists so do that now
-        const primaries = parseStringList(withJoiner.primary, delims);
+        const primaries = parseContextAwareStringList(withJoiner.primary, delims, {ignoreGlobalAmpersand: ignoreGlobalAmpersand ?? false});
        if (primaries.length > 1) {
            return {
                primary: primaries[0],
@@ -159,7 +159,7 @@ export const parseArtistCredits = (str: string, delimiters?: boolean | string[])
        return withJoiner;
    }
    // likely this is a plain string with just delims
-    const artists = parseStringList(str, delims);
+    const artists = parseContextAwareStringList(str, delims, {ignoreGlobalAmpersand: ignoreGlobalAmpersand ?? true});
    if (artists.length > 1) {
        return {
            primary: artists[0],
@@ -173,7 +173,7 @@ export const parseArtistCredits = (str: string, delimiters?: boolean | string[])
    }
 }
 export const parseTrackCredits = (str: string, delimiters?: boolean | string[]): PlayCredits | undefined => parseCredits(str, delimiters);
-export const parseStringList = (str: string, delimiters: string[] = [',', '&', '/', '\\']): string[] => {
+export const parseStringList = (str: string, delimiters: string[] = DELIMITERS): string[] => {
    if (delimiters.length === 0) {
        return [str];
    }
@@ -182,10 +182,54 @@ export const parseStringList = (str: string, delimiters: string[] = [',', '&', '
        return explodedStrings.flat(1);
    }, [str]).map(x => x.trim());
 }
+export const parseContextAwareStringList = (str: string, delimiters: string[] = [',', '/', '\\'], opts: {ignoreGlobalAmpersand?: boolean} = {}): string[] => {
+    if (delimiters.length === 0) {
+        return [str];
+    }
+    // bypass tokens using slashes without spaces
+    const cleanStr = bypassJoiners(str);
+    const nonAmpersandDelims = delimiters.some(x => cleanStr.includes(x));
+    const shouldIgnoreGlobalAmpersand = opts.ignoreGlobalAmpersand ?? nonAmpersandDelims;
+
+    let awareList: string[] = [];
+
+    const list = parseStringList(cleanStr, nonAmpersandDelims === false && shouldIgnoreGlobalAmpersand === false ? ['&'] : delimiters);
+    if(shouldIgnoreGlobalAmpersand && list.length > 1 && list[list.length - 1].includes('&') && nonAmpersandDelims) { //&& !list[list.length - 1].includes('& the')
+        awareList = list.slice(0, list.length - 1).concat(list[list.length - 1].split('&') );
+    } else {
+        awareList = list;
+    }
+    return awareList.map(x =>rejoinBypassed(x.trim()));
+}
+
+const bypassJoinerMap = [
+    {
+        rejoin: str => str.replaceAll(/(.*?\S)(\^\^\^)(\S.*?)/g, '$1/$3'),
+        bypass: str => str.replaceAll(/(.*?\S)(\/)(\S.*?)/g, '$1^^^$3')
+    },
+    {
+        rejoin: str => str.replaceAll(/(.*)(###)(.*)/g, '$1\\$3'),
+        bypass: str => str.replaceAll(/(.*\S)(\\)(.*\S)/g, '$1###$3')
+    }
+];
+export const bypassJoiners = (str: string): string => {
+    let bypassed: string = str;
+    for(const b of bypassJoinerMap) {
+        bypassed = b.bypass(bypassed)
+    }
+    return bypassed;
+}
+export const rejoinBypassed = (str: string): string => {
+    let bypassed: string = str;
+    for(const b of bypassJoinerMap) {
+        bypassed = b.rejoin(bypassed)
+    }
+    return bypassed;
+}
 export const containsDelimiters = (str: string) => null !== str.match(/[,&/\\]+/i)
-export const findDelimiters = (str: string) => {
+export const findDelimiters = (str: string, delimiters = DELIMITERS) => {
    const found: string[] = [];
-    for (const d of DELIMITERS) {
+    for (const d of delimiters) {
        if (str.indexOf(d) !== -1) {
            found.push(d);
        }
@@ -5,7 +5,31 @@ import { AdditionalTrackInfoResponse } from "../backend/common/vendor/listenbrai

 export interface SourceStatusData {
    status: string;
-    type: "spotify" | "plex" | "tautulli" | "subsonic" | "jellyfin" | "lastfm" | "deezer" | "ytmusic" | "mpris" | "mopidy" | "listenbrainz" | "jriver" | "kodi" | 'webscrobbler' | 'chromecast';
+    type: 'spotify'
+    | 'plex'
+    | 'tautulli'
+    | 'subsonic'
+    | 'jellyfin'
+    | 'lastfm'
+    | 'deezer'
+    | 'endpointlz'
+    | 'endpointlfm'
+    | 'ytmusic'
+    | 'mpris'
+    | 'mopidy'
+    | 'musiccast'
+    | 'listenbrainz'
+    | 'jriver'
+    | 'kodi'
+    | 'webscrobbler'
+    | 'chromecast'
+    | 'maloja'
+    | 'musikcube'
+    | 'mpd'
+    | 'vlc'
+    | 'icecast'
+    | 'azuracast'
+    | 'koito';
    display: string;
    tracksDiscovered: number;
    name: string;
@@ -23,7 +47,7 @@ export interface SourceStatusData {

 export interface ClientStatusData {
    status: string;
-    type: "maloja" | "lastfm" | "listenbrainz";
+    type: "maloja" | "lastfm" | "listenbrainz" | "koito";
    display: string;
    scrobbled: number;
    deadLetterScrobbles: number
@@ -52,17 +76,26 @@ export interface TrackStringOptions<T = string> {
    }
 }

-export interface PlayProgress {
-    timestamp: Dayjs
+export interface PlayProgressAmb {
+    timestamp: string | Dayjs
    position?: number
    positionPercent?: number
 }

+export interface PlayProgress extends PlayProgressAmb {
+    timestamp: Dayjs
+}
+
 export interface PlayProgressPositional extends PlayProgress {
    position: number
 }

-export interface ListenRangeData {
+export interface ListenRangeDataAmb {
+    start: PlayProgressAmb
+    end: PlayProgressAmb
+}
+
+export interface ListenRangeData extends ListenRangeDataAmb {
    start: ListenProgress
    end: ListenProgress
 }
@@ -343,4 +376,13 @@ export interface URLData {
    url: URL
    normal: string
    port: number
-}
+}
+
+export type Joiner = ',' | '&' | '/' | '\\' | string;
+export const JOINERS: Joiner[] = [',','/','\\'];
+
+export type FinalJoiners = '&';
+export const JOINERS_FINAL: FinalJoiners[] = ['&'];
+
+export type Feat = 'ft' | 'feat' | 'vs' | 'ft.' | 'feat.' | 'vs.' | 'featuring'
+export const FEAT: Feat[] = ['ft','feat','vs','ft.','feat.','vs.','featuring'];
@@ -201,3 +201,29 @@ export const combinePartsToString = (parts: any[], glue: string = '-'): string |
    }
    return undefined;
 }
+
+export const arrayListOxfordAnd = (list: string[], joiner: string, finalJoiner: string, spaced: boolean = true): string => {
+    if(list.length === 1) {
+        return list[0];
+    }
+    const start = list.slice(0, list.length - 1);
+    const end = list.slice(list.length - 1);
+
+    const joinerProper = joiner === ',' ? ', ' : (spaced ? ` ${joiner} ` : joiner);
+    const finalProper = spaced ? ` ${finalJoiner} ` : finalJoiner;
+
+    return [start.join(joinerProper), end].join(joiner === ',' && spaced ? `,${finalProper}` : finalProper);
+}
+
+export const arrayListAnd = (list: string[], joiner: string, finalJoiner: string, spaced: boolean = true): string => {
+    if(list.length === 1) {
+        return list[0];
+    }
+    const start = list.slice(0, list.length - 1);
+    const end = list.slice(list.length - 1);
+
+    const joinerProper = joiner === ',' ? ', ' : (spaced ? ` ${joiner} ` : joiner);
+    const finalProper = spaced ? ` ${finalJoiner} ` : finalJoiner;
+
+    return [start.join(joinerProper), end].join(finalProper);
+}
Author	SHA1	Message	Date
FoxxMD	8cdf779d87	feat(lzendpoint): Add date/listened during MS specific properties #360	2025-09-19 13:06:23 +00:00
FoxxMD	44f2a295ff	feat(lzendpoint): Add MS specific timestamp SOC to payload #360	2025-09-19 12:54:20 +00:00
FoxxMD	e061540e6b	fix(jellyfin): Guard against undefined NowPlaying Path #355	2025-09-18 14:15:20 +00:00
FoxxMD	38bf31a67f	feat(lzendpoint): Accept artist and release artist names in submit payload #360	2025-09-18 13:45:29 +00:00
Matt Foxx	c8c3b4888f	Merge pull request #343 from FoxxMD/caching feat: Caching and persistent scrobble queues	2025-09-17 08:54:44 -04:00
FoxxMD	70bb30406a	test: fix/workaround tests for new transformrule assumptions	2025-09-16 15:27:47 +00:00
FoxxMD	d9e71f6770	fix: Do not use transform until after client is ready Fixes #356	2025-09-16 15:16:27 +00:00
FoxxMD	0a673fd333	refactor: Decrease persist interval to 10 seconds Faster interval means less likely user will update queue and exit MS before cache is saved	2025-09-11 19:14:01 +00:00
FoxxMD	fd316a8e84	fix: Add full coverage for cached queue updating during client lifecycle	2025-09-11 19:14:01 +00:00
FoxxMD	c1e1de2255	Increase file cache size to 2000 Should be ~700kb, full up. This should be a generous upper limit for number of scrobbles queued/dead at the same time.	2025-09-11 19:14:01 +00:00
FoxxMD	9bea31e1b2	docs: Add caching docs	2025-09-11 19:14:01 +00:00
FoxxMD	351edd8102	fix: Fix cache test missing awaits	2025-09-11 19:14:01 +00:00
FoxxMD	10dcb7f30d	feat: Implement cache file streaming with progress logging If cache files get large enough to cause noticeable delay it will be helpful to log progress	2025-09-11 19:14:01 +00:00
FoxxMD	ffb41b4e30	test: Add scrobble client caching test	2025-09-11 19:14:01 +00:00
FoxxMD	d78bc48e6f	feat: Allow passing file cache options other than connection	2025-09-11 19:14:01 +00:00
FoxxMD	b4154bb0d5	refactor: Move transient cache to exportable function	2025-09-11 19:14:01 +00:00
FoxxMD	48467b08d3	chore: Bump with-local-tmp-dir version tmp dependency has been removed so previous buggy behavior(?) is gone	2025-09-11 19:14:01 +00:00
FoxxMD	682d507683	build: move net-interceptor to dev packages	2025-09-11 19:14:01 +00:00
FoxxMD	814e609efc	fix: Add clone as runtime dependency	2025-09-11 19:14:01 +00:00
FoxxMD	652fa5a14c	feat(cache): Improve data marshalling and cache init * Use typeson for stringified data marshalling * Use clone for in-memory serializing to preserve dayjs * Refactor cache backend init into indepedent, testable functions * Add `false` as cache provider type to allow disabling secondary cache	2025-09-11 19:14:01 +00:00
FoxxMD	39bc9cd410	build(vscode): Update devcontainer to use compose stack with valkey for caching integration	2025-09-11 19:14:01 +00:00
FoxxMD	63b6012488	feat(cache): Rehydrate dayjs data on cached scrobble parsing	2025-09-11 19:14:01 +00:00
FoxxMD	32e2aa6974	test: Setup transient di cache	2025-09-11 19:14:01 +00:00
FoxxMD	78f4767dd2	refactor: Refactor di cache to allow singleton or transient	2025-09-11 19:14:01 +00:00
FoxxMD	05a9ca15a9	refactor(cache): Update file cache name and gitignore	2025-09-11 19:14:01 +00:00
FoxxMD	e2c8158aec	feat: Log file cache save	2025-09-11 19:14:01 +00:00
FoxxMD	4eee6d4322	somewhat working scrobble caching	2025-09-11 19:14:01 +00:00
FoxxMD	56f5431dd9	chore: Update iti usage * Update package to improve typing support/esm * Remove dependency on sources/client/notifiers to prevent circular dependency	2025-09-11 19:14:01 +00:00
FoxxMD	a00f94f9b1	feat: Implement caching backend * Two-tier caching * primary is always in-memory LRU * secondary supports file and valkey * Different caches for metadata and scrobbles * no file support for metadata (too big)	2025-09-11 19:14:01 +00:00
Matt Foxx	df4fe87178	Merge pull request #351 from FoxxMD/ntfyToken feat: Add Access Token auth for Ntfy webhook	2025-09-11 15:07:11 -04:00
Matt Foxx	89c39d2d25	Merge pull request #354 from FoxxMD/lastfmRecent fix(lastfm): Full destructure recent response and log respon on error	2025-09-11 15:01:41 -04:00
FoxxMD	b60422c26c	fix: Try auth prefixed with Bearer	2025-09-11 18:57:30 +00:00
FoxxMD	59fce8271e	feat: Add Access Token auth for Ntfy webhook	2025-09-11 18:57:30 +00:00
FoxxMD	506faf39dc	fix(lastfm): Full destructure recent response and log respon on error Maybe recenttracks/track does not exist for totally new accounts, though it should. Guard against this and log response if something unexpected happens so we can troubleshoot #353	2025-09-11 18:27:56 +00:00
Matt Foxx	adf5401fff	Merge pull request #347 from FoxxMD/ytm400 Youtube Music updates	2025-09-10 10:15:55 -04:00
FoxxMD	dbf5cba7cb	chore: Bump logging version Fixes LOG_LEVEL non-output issue	2025-09-10 13:04:34 +00:00
FoxxMD	b3a6beb8f4	docs(ytm): Add oauth warning	2025-09-10 12:57:42 +00:00
Matt Foxx	1291f60e62	Merge pull request #260 from FoxxMD/improvedArtistStringParsing testing and improvements for artist string parsing	2025-09-10 08:47:50 -04:00
FoxxMD	e86945355c	fix: Add additional logging for ytm history request	2025-09-05 18:56:16 +00:00
FoxxMD	8dd9958b30	Use default scopes	2025-09-05 18:55:53 +00:00
FoxxMD	33fda80d0a	feat: Improve artist string parsing * Don't split artist with joiner when only one joiner is present * Override naive parsing if metabrainz mapped artists fit with provided joiners	2025-08-07 19:32:51 +00:00
FoxxMD	a3d8372f48	feat: More artist parsing improvements * Use more context-aware list parsing with ampersands * Ignore artists with slashes when wrapped by word-boundary	2025-08-07 13:23:10 +00:00
FoxxMD	b5dbde66c4	WIP testing and improvements for artist astring parsing * Implement faker data generation for artist string * Add test suite for parsing different styles of artist strings	2025-08-07 13:23:10 +00:00