Compare commits

...
Author SHA1 Message Date
FoxxMD 6f8e7d3862 docs: Update intallation instructions with config/data dir guidance
#616
2026-07-16 16:14:24 -04:00
FoxxMD a6d33fc2ef feat: Move config and data dir creation into app
* only use docker init for fixing permission
  * don't rely on docker init for dir creation as other distros may need this functionality (nix)
* check for and create config/data dirs recursively on app start

#616
2026-07-16 16:14:24 -04:00
FoxxMD 26121ace10 feat: Separate config and data directories and use OS-standard paths, if present
* Break configDir into config and data directories so config and database/cache can be stored in different directories
* Use `env-paths` to fallback to OS-standard paths for config and data if CONFIG_DIR and DATA_DIR envs are not present
* Additionally fallback to process CWD/config if none of the above are present for some reason

#616
2026-07-16 16:14:24 -04:00
FoxxMD 27e5656a04 refactor: Replace projectDir with explicit path to project root, projectRootDir 2026-07-16 16:14:24 -04:00
Matt Foxx f3d7d59af6 Merge pull request #636 from Exerra/feat/apple-music
feat: Apple Music source
2026-07-16 15:45:04 -04:00
FoxxMD 2a4afefe64 feat(applemusic): Make album normalization user configurable and add guidance on usage 2026-07-16 16:41:36 +00:00
FoxxMD 1dba73ccb2 feat(client)!: Overhaul source scrobble filtering
* Use id OR name for matching client to scrobble to, during transition period
* Deprecate using name-only and warn when this is detected
* Consolidate logging filtered clients to one statement to reduce log noise
* Improve filter matching by making strings case and whitepsace insensitive
* On startup log `clients` options for sources
2026-07-16 15:51:54 +00:00
FoxxMD 3e82e035ac feat: Update string normalization with granular options 2026-07-16 15:49:01 +00:00
Exerra b12153fba9 docs(applemusic): improve formatting of developer tools shortcut
Updated the Apple Music configuration documentation to use <kbd> tags for keyboard shortcuts, improving readability.
2026-07-16 18:12:33 +03:00
Exerra 8c82412de4 docs: add detailed Apple Music authorization token extraction guide
Updated Apple Music configuration documentation with step-by-step instructions and screenshots for extracting authorization tokens in Chrome, Edge, Firefox, and Safari.
2026-07-16 17:15:58 +03:00
FoxxMD e804ce33ef fix(listenbrainz): Fix play fetch more indicator
latest_listen_ts is *always* the latest of *all-time*, not for the current min/max_ts range requested.

Instead, use count < than desired (and non-zero) to indicate more results
2026-07-15 20:49:37 +00:00
FoxxMD 3907fa5474 fix: Fix play fetch options logging data for pageless functions 2026-07-15 20:48:32 +00:00
FoxxMD a7b3593314 stage import rule from eslint-plugin-import-x 2026-07-15 19:08:48 +00:00
FoxxMD 76ef994067 fix: Implement unicorn eslint rules and fix them 2026-07-15 18:31:13 +00:00
FoxxMD c3cb3cf3a1 fix(client): Correct SOT scrobble api requests to not have the same from/to value 2026-07-15 17:34:37 +00:00
FoxxMD 8096219526 fix(client): Correct create/update timestamps for queues and scrobbles
* Ensure ingress queue always has an updated timestamp before dead creation
* Don't re-assign createdAt for scrobble after dead queue has already been updated
2026-07-15 17:02:29 +00:00
FoxxMD 67c1823195 feat(ui): Clean up scrobble timeline items
* Use timeline summary formatting to fix spacing
* Provide client name from fetched component data, if present, so scrobble summary is explicit where it sent/response scrobble from
2026-07-15 16:22:04 +00:00
FoxxMD 2014e78ade fix(ui): Stabilize play timeline item order 2026-07-15 14:51:42 +00:00
Exerra b3c332deeb refactor: replace Apple Music headers config with origin field
Replaces the generic `headers` configuration field with a specific `origin` field for Apple Music. Updates the config schema, environment variable handling, documentation, and source implementation to simplify origin header management.
2026-07-15 17:12:21 +03:00
Exerra 93a1042e03 test(applemusic): add tests for stripping EP/Single album suffixes
Added test cases to ensure that ' - EP' and ' - Single' suffixes are correctly stripped from Apple Music album names during the formatting process.
2026-07-15 16:21:10 +03:00
Exerra ba010240c0 feat(AppleMusicSource): include track URL in metadata
Added the web URL to the track metadata in AppleMusicSource to provide a direct link to the track.
2026-07-15 16:20:54 +03:00
FoxxMD 7f491d6974 fix: Fix hash router type 2026-07-15 13:08:04 +00:00
Exerra 74e67e0e41 feat(AppleMusicSource): include ISRC in track metadata
Added the isrc field to the track object in AppleMusicSource to improve track identification accuracy.
2026-07-15 15:59:49 +03:00
FoxxMD 1853f9abba fix(musicbrainz): Catch scenario where all search prereqs fail
#636
2026-07-15 12:58:40 +00:00
Exerra c13e7ebeec feat(applemusic): strip ' - EP' and ' - Single' from album names
Implemented album name normalization in AppleMusicSource to ensure consistency with other sources by removing standard suffixes from EPs and singles. Updated documentation to reflect this behavior.
2026-07-15 15:38:14 +03:00
Exerra 207354e799 docs(docsite): fix heading hierarchy in applemusic.mdx
Adjusted markdown heading levels to ensure proper document structure.
2026-07-15 15:36:51 +03:00
FoxxMD d7f6b17308 update boundary element linting to be more lenient
* allow core-tests from backend
* downgrade linting to warn
2026-07-15 12:23:53 +00:00
Exerra c069020aac docs(README.md): add Apple Music to supported sources list
Updated the README to include Apple Music in the list of supported scrobbling sources with a link to its documentation.
2026-07-15 13:52:53 +03:00
Exerra f84a0f190d test(applemusic): add integration tests for Apple Music module
Add new test suite for Apple Music backend logic
2026-07-15 13:39:27 +03:00
FoxxMD d0f43fbd2d feat(ui): Increase virtual play list height 2026-07-15 01:29:12 +00:00
FoxxMD b38168975e feat(ui): Compact filters
* move refresh/loading indicator into list type header
* Remove filter card title to trim filter real estate
2026-07-15 01:22:40 +00:00
Exerra a845ac10ce feat(applemusic): implement top-rebound recovery for duplicate plays
Added logic to detect and recover 'top-rebound' history patterns where Apple Music deduplicates re-listened tracks by bumping them to the top. This ensures intermediate tracks are captured and re-listened top tracks are processed correctly. Includes a new configuration option `recoverUnchangedTopHistory` to toggle this behavior.
2026-07-15 02:44:41 +03:00
Exerra 56fdb12683 feat(sources): update Apple Music configuration and environment variables
Refactored Apple Music source configuration to support token-based authentication via key files and custom headers. Renamed environment variable prefixes from 'AM' to 'APPLEMUSIC' for consistency.
2026-07-15 00:04:14 +03:00
Exerra 1f40e1e414 docs(applemusic): update configuration guide and add APPLEMUSIC_HEADERS
Added documentation for the APPLEMUSIC_HEADERS environment variable and updated code blocks with titles for better clarity in the Apple Music source configuration.
2026-07-15 00:03:06 +03:00
Exerra c1a260dc56 fix(AppleMusicSource): add authentication verification to connection checks
Implemented status code validation for Apple Music authentication and connection checks. The module now throws errors instead of failing silently when the API is unreachable or auth is invalid.
2026-07-14 21:56:59 +03:00
Exerra 5d34ea68af docs: update Apple Music configuration guide
Reorganized the Apple Music documentation to prioritize the Browser Token (Free) authentication method and clarify requirements for the paid MusicKit Key approach.
2026-07-14 21:27:23 +03:00
FoxxMD 41ea9e0aa7 feat(ui): Make timeline items more compact
#500
2026-07-14 15:23:37 +00:00
FoxxMD a9d91c2c8a fix circular buffer 2026-07-14 14:04:19 +00:00
FoxxMD a696436a75 fix: Fix missing early return for skipped stage 2026-07-14 13:47:57 +00:00
Exerra 4ae388676e chore(package-lock.json): update dependencies
Bump node-musickit-api to 3.1.0-next.5
2026-07-14 14:12:42 +03:00
Exerra 89b698d48d docs: add Apple Music source documentation
Add Apple Music to the sources overview table and the main documentation index.
2026-07-14 14:06:20 +03:00
Exerra b71a4cbbd6 build(package.json): bump node-musickit-api to v3.1.0-next.5
Updated node-musickit-api dependency to the latest next version.
2026-07-14 13:08:36 +03:00
Exerra a870cdc6f4 chore(package.json): update node-musickit-api to v3.1.0-next.4
Switched node-musickit-api dependency from a local path to version 3.1.0-next.4 from the registry.
2026-07-14 01:11:29 +03:00
Exerra c404dd5396 feat(applemusic): add support for custom headers in API requests
Introduced a `headers` configuration option to allow setting custom request headers, such as 'Origin', for Apple Music API requests. Updated configuration schema and source implementation to apply these headers.
2026-07-14 01:03:26 +03:00
Exerra cae845028d refactor(AppleMusicSource): use HistoryConsistencyResult type alias
Update parseRecentAgainstResponse return type signature to use the HistoryConsistencyResult alias instead of an inline object definition.
2026-07-14 00:50:58 +03:00
Exerra 1698c5d71e refactor(AppleMusicSource): clarify timestamp calculation logic
Updated comments in applyCalculatedTimestamps to reflect that Apple Music lacks track timestamps, necessitating a hacky estimation method.
2026-07-14 00:48:29 +03:00
Exerra 765d86e02b refactor(AppleMusicSource): extract timestamp calculation to method
Moved the reduceRight logic for calculating play timestamps into a dedicated private method 'applyCalculatedTimestamps' to improve code readability and maintainability.
2026-07-14 00:46:58 +03:00
Exerra f347b82d33 refactor(AppleMusicSource): remove redundant unknown type assertion
Cleaned up the type casting in the AppleMusicSource fetch logic by removing unnecessary unknown assertion.
2026-07-14 00:44:38 +03:00
Exerra 87aa9c461e refactor(AppleMusicSource): replace magic numbers with class constants
Introduced constants for history limits and track polling constraints to improve maintainability and readability within AppleMusicSource.
2026-07-14 00:44:25 +03:00
Exerra 417ce85bc7 refactor(AppleMusicSource): improve type safety and code readability
Refactored AppleMusicSource to use defined types for history consistency, updated the getRecentlyPlayedTracks limit type, and renamed variable for clarity.
2026-07-14 00:42:01 +03:00
Exerra c8d917cfe6 refactor(AppleMusicSource): replace AppleMusicTrack interface with Song type
Removed local AppleMusicTrack interface and updated formatPlayObj to utilize the shared Song type for improved type consistency.
2026-07-14 00:34:47 +03:00
Exerra 8c789123f9 refactor(AppleMusicSource): remove unused backlogged plays logic
Removed the unused `getBackloggedPlays` method and cleaned up the `getRecentlyDiscoveredPlays` logic in `AppleMusicSource` to streamline track processing.
2026-07-14 00:31:06 +03:00
Exerra 06ed3f4388 feat: add Apple Music source support
Implement Apple Music source, including configuration types, environment variable handling, and source initialization logic. Refactored SourceType definitions to improve maintainability.
2026-07-14 00:24:44 +03:00
FoxxMD fab2e4938d feat(ui): Persist log window state #500
Store opened state, and position/size, to local storage so logs stay where user wants them across refreshes/initial load.

Resets window to default position if any part of log window is outside viewport bounds on open.
2026-07-12 16:19:13 +00:00
FoxxMD fbcd1e1b9a refactor(ui): Fix conditional hook for SSE and add stories for header elements 2026-07-12 15:43:33 +00:00
FoxxMD d48f2bd4ac feat(ui): Replace now playing player with summary with reduced motion #500 2026-07-11 20:28:13 +00:00
FoxxMD e2dad463f6 feat: reduce now playing animation #500 2026-07-11 20:05:20 +00:00
FoxxMD 642af13f3c fix(ui): fix shiki instance creation 2026-07-10 21:26:51 +00:00
FoxxMD 4146b159bd feat(ui): Component list container max width based on selected grid width 2026-07-10 21:11:13 +00:00
FoxxMD 3a3a97132d fix jellyfin import? 2026-07-10 21:03:06 +00:00
FoxxMD 04a5b91694 feat(ui): Implement grid for desktop layout
Allow user-selected max grid width on non-mobile breakpoints

#500
2026-07-10 20:57:03 +00:00
FoxxMD 945f494a2e fix: top-level type import syntax
see eslint.config.js comments
2026-07-10 16:47:38 +00:00
FoxxMD 4b09909b33 update cacheable for per-operation ttl 2026-07-10 15:17:01 +00:00
FoxxMD a5a1365b72 chore: remove tsx from mocha extension 2026-07-10 15:16:42 +00:00
FoxxMD 6bc618c481 always cache mb url so we can log to transform input 2026-07-10 15:16:31 +00:00
FoxxMD 85d1a0e648 chore: fix code styling in musicbrainz transformer 2026-07-10 15:15:57 +00:00
FoxxMD 1ae45c6060 add npm script for generating esbuild bundle metadata 2026-07-10 13:34:15 +00:00
FoxxMD 7daa43a883 tree-shake shiki for chakra usage 2026-07-10 13:34:15 +00:00
FoxxMD f8501b2a2e chore: Remove tsx usage from vscode launch profiles 2026-07-10 13:34:15 +00:00
Matt Foxx 14c35edadf Merge pull request #633 from FoxxMD/node-ts-native
feat: Migrate to native node runtime everywhere
2026-07-09 16:25:27 -04:00
302 changed files with 3909 additions and 1744 deletions
+5 -1
View File
@@ -155,4 +155,8 @@ storybook-static
*.db
*.db*
**/lexicons/**/*.json
**/lexicons/**/*.json
bun.lock
.DS_Store
**/.DS_Store
+5 -20
View File
@@ -20,7 +20,7 @@
"request": "launch",
// Debug app in VSCode
"program": "${workspaceFolder}/src/backend/index.ts",
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"runtimeExecutable": "node",
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"skipFiles": [
@@ -35,7 +35,7 @@
"request": "launch",
// Debug app in VSCode
"program": "${workspaceFolder}/src/backend/index.ts",
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"runtimeExecutable": "node",
"console": "integratedTerminal",
"envFile": "${workspaceFolder}/.env",
"env": {
@@ -48,28 +48,13 @@
"${workspaceFolder}/node_modules/**",
],
},
{
"name": "tsx",
"type": "node",
"request": "launch",
// Debug current file in VSCode
"program": "${file}",
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"skipFiles": [
"<node_internals>/**",
// Ignore all dependencies (optional)
"${workspaceFolder}/node_modules/**",
],
},
{
"name": "schema",
"type": "node",
"request": "launch",
// Debug app in VSCode
"program": "${workspaceFolder}/src/backend/utils/SchemaStaticUtil.ts",
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"runtimeExecutable": "node",
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"skipFiles": [
@@ -96,7 +81,7 @@
"--recursive",
"${workspaceFolder}/src/backend/tests/**/*.test.ts"
],
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"runtimeExecutable": "node",
"internalConsoleOptions": "openOnSessionStart",
"name": "Mocha Tests",
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
@@ -120,7 +105,7 @@
"--config", "${workspaceRoot}/.mocharc.json",
"${file}"
],
"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/tsx",
"runtimeExecutable": "node",
"internalConsoleOptions": "openOnSessionStart",
"name": "Mocha Test on File",
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
-1
View File
@@ -1,5 +1,4 @@
{
"mochaExplorer.require": "tsx/esm",
"mochaExplorer.timeout": 1200000,
"mochaExplorer.exit": true,
"search.exclude": {
+1
View File
@@ -56,6 +56,7 @@ RUN npm install -g concurrently
ARG data_dir=/config
VOLUME $data_dir
ENV CONFIG_DIR=$data_dir
ENV DATA_DIR=$data_dir
COPY docker/root /
+1
View File
@@ -12,6 +12,7 @@ alt="multi-scrobbler logo" width="180" height="180">
A dockerized app that monitors your music listening activity from *everywhere* and scrobbles it *anywhere*.
* Supports scrobbling from many [**Sources**](https://docs.multi-scrobbler.app/configuration/sources)
* [Apple Music](https://docs.multi-scrobbler.app/configuration/sources/applemusic/?configType=file)
* [Azuracast](https://docs.multi-scrobbler.app/configuration/sources/azuracast)
* [Deezer](https://docs.multi-scrobbler.app/configuration/sources/deezer)
* [Google Cast (Chromecast)](https://docs.multi-scrobbler.app/configuration/sources/google-cast)
+29
View File
@@ -0,0 +1,29 @@
[
{
"id": "myAppleMusic",
"name": "My Apple Music",
"enable": true,
"clients": [],
"data": {
// either key or token needs to be provided
"key": {
"id": "2HPSNJZ88N",
"teamId": "SN6YASW8G4",
"p8": "-----BEGIN PRIVATE KEY-----\nMIGTAgEAMBMGByqGSM49AgEG...-----END PRIVATE KEY-----"
},
"token": "your-jwt-token-here",
"mediaUserToken": "your-media-user-token-here",
// optional: Origin header for API requests (required when using a browser token)
"origin": "https://music.apple.com",
// optional: polling interval in seconds (default 60)
"interval": 60
},
"options": {
"logDiff": true,
"recoverUnchangedTopHistory": true,
"normalizeAlbum": true
}
}
]
@@ -1,33 +1,19 @@
#!/usr/bin/with-contenv bash
# used https://github.com/linuxserver/docker-plex as a template
POPULATE_EXAMPLES=false
echo "-------------------------------------"
echo -e "Setting up multi-scrobbler config directory based on CONFIG_DIR env: ${CONFIG_DIR}\n"
echo -e "Verifying multi-scrobbler directory permissions based on CONFIG_DIR env: ${CONFIG_DIR} and DATA_DIR env: ${DATA_DIR}\n"
# make config folder if it does not exist
if [ ! -d "${CONFIG_DIR}" ]; then
echo "Directory does not exist! Creating..."
POPULATE_EXAMPLES=true
mkdir -p "${CONFIG_DIR}"
else
if [ "$(ls -A ${CONFIG_DIR})" ]; then
echo "Directory is not empty, not creating examples."
else
POPULATE_EXAMPLES=true
fi
if [ -d "${CONFIG_DIR}" ]; then
echo "chown'ing config directory to ensure correct permissions."
chown -R abc:abc "${CONFIG_DIR}"
echo "Done!"
fi
# add example configs
if [ "$POPULATE_EXAMPLES" = true ]; then
echo "Directory is empty, adding examples..."
cp -r /app/config/. "${CONFIG_DIR}"/
if [ -d "${DATA_DIR}" ]; then
echo "chown'ing data directory to ensure correct permissions."
chown -R abc:abc "${DATA_DIR}"
echo "Done!"
fi
# permissions
echo "chown'ing directory to ensure correct permissions."
chown -R abc:abc "${CONFIG_DIR}"
echo "Done!"
echo -e "-------------------------------------\n"
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 501 KiB

@@ -0,0 +1,262 @@
---
title: Apple Music
toc_min_heading_level: 2
toc_max_heading_level: 5
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';
import AppleMusicConfig from '!!raw-loader!@site/../config/applemusic.json.example';
This Source **monitors your Apple Music listening history** via the official Apple Music API and scrobbles new activity to your configured [Clients](/configuration/clients).
Because of how the Apple Music API works, Multi-Scrobbler (MS) requires two different tokens to function: a **Media User Token** (identifies your personal account) and an **Authentication Token** (authorizes API access).
### 1. Getting a Media User Token
The `mediaUserToken` is required to access your recently played history. You can extract it directly from the Apple Music web app:
1. Visit [music.apple.com](https://music.apple.com) in your browser and log in.
2. Open your browser's Developer Tools (usually `F12` or <kbd>⌘</kbd> + <kbd>⌥</kbd> + <kbd>I</kbd>).
3. Navigate to the **Application** tab (Chrome/Edge) or **Storage** tab (Safari/Firefox).
4. Expand **Cookies** in the left sidebar and select `https://music.apple.com`.
5. Find the cookie named `media-user-token` and copy its value (it typically starts with `0.`).
:::caution[Token Expiry]
The `mediaUserToken` will eventually expire. If Multi-Scrobbler begins throwing authentication errors in the logs, simply repeat these steps to obtain and configure a fresh token.
:::
### 2. Authentication
To authorize API access, Multi-Scrobbler requires an Authentication Token (JWT). Because the official Apple Music API is designed for developers, there are two ways to provide this token depending on your situation:
* **Browser Token (Free & Most Common):** Best for 99% of users. You can easily extract a temporary token from the Apple Music web player without needing an Apple Developer account.
* **MusicKit Key (Requires Paid Apple Developer Account):** If you happen to be enrolled in the paid Apple Developer Program ($99/year), you can provide a private key to automatically generate permanent tokens.
Choose your preferred method below:
<Tabs groupId="appleMusicAuth" queryString>
<TabItem value="token" label="Browser Token (Free)">
Since most users don't have a paid developer account, extracting the token from the web player is the standard approach.
Behind the scenes, the Apple Music website makes a lot of background requests, which can make finding the right token confusing. To filter out the noise and grab the exact token we need, follow the steps for your browser below:
<details>
<summary>Chrome, Edge & Firefox</summary>
1. Visit [music.apple.com](https://music.apple.com) and log in.
2. Open Developer Tools (`F12` or `Ctrl+Shift+I` / <kbd>⌘</kbd> + <kbd>⌥</kbd> + <kbd>I</kbd>) and navigate to the **Network** tab.
3. In the filter box at the top left, paste exactly: `https://amp-api.music.apple.com/v1/me/account` (**Label 1**).
4. Click the **Fetch/XHR** filter button to hide irrelevant requests (**Label 2**).
5. Reload the page (**Label 3**).
6. Under the "Name" column, click the request starting with `account?meta=` (**Label 4**).
7. In the panel that opens, scroll down to the **Request Headers** section.
8. Find the `authorization` header. Copy the long string of text **after** the `Bearer ` prefix (**Label 5**).
![Chrome Network Tab Instructions](../applemusic_chrome_devtools.png)
</details>
<details>
<summary>Safari</summary>
Before you can extract the token in Safari, you must enable developer tools.
{/* This weird formatting is because ⌘ is usually incredibly tiny */}
1. Open Safari Settings (<kbd>⌘</kbd> + <kbd>,</kbd>), navigate to the **Advanced** tab, and check **"Show features for web developers"** at the very bottom.
![Safari Advanced Settings](../applemusic_safari_settings.png)
2. Visit [music.apple.com](https://music.apple.com) and log in.
3. Open the Web Inspector (<kbd>⌘</kbd>+<kbd>⌥</kbd>+<kbd>I</kbd> or Develop > Show Web Inspector) and navigate to the **Network** tab.
4. In the filter bar on the left side of the inspector, paste exactly: `https://amp-api.music.apple.com/v1/me/account` (**Label 1**).
5. Reload the page (**Label 2**).
6. Click the `account` request that appears in the Name list (**Label 3**).
7. In the details sidebar, look under the **Request** section for the `Authorization` header. Copy the long string of text **after** the `Bearer ` prefix (**Label 4**).
![Safari Network Tab Instructions](../applemusic_safari_devtools.png)
</details>
<br />
Once you have your token, add it to your configuration file along with the origin header:
```json title="applemusic.json"
{
"data": {
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsIm...",
"mediaUserToken": "your-media-user-token-here",
"origin": "https://music.apple.com"
}
}
```
:::info[Origin Header Required]
When using a JWT extracted from the browser, Apple requires the request to match the domain it was issued to. You **must** include the `origin` field in your config as shown above.
:::
*Note: Browser-generated JWTs are valid for a maximum of 35 days and must be updated manually when they expire.*
</TabItem>
<TabItem value="key" label="MusicKit Key (Paid Account)">
:::warning[Apple Developer Account Required]
To generate JWTs automatically, you must be enrolled in the paid [Apple Developer Program](https://developer.apple.com/programs/) ($99/year) and create a MusicKit key. If you don't have an account, use the **Browser Token (Free)** tab instead.
:::
To generate JWTs automatically and avoid manual token refreshes, you need an Apple Music API key:
1. Go to the [Apple Developer portal](https://developer.apple.com/account/resources/authkeys/list) and create a **MusicKit** key.
2. Download the `.p8` file — this is your private key.
3. Note your **Key ID** and **Team ID** (found in your Apple Developer account Membership details).
Add these details to your config:
```json title="applemusic.json"
{
"data": {
"key": {
"id": "2HPSNJZ88N",
"teamId": "SN6YASW8G4",
"p8": "-----BEGIN PRIVATE KEY-----\nMIGTAgEAMBMGByqGSM49AgEG...-----END PRIVATE KEY-----"
},
"mediaUserToken": "your-media-user-token-here"
}
}
```
:::tip
When pasting the contents of your `.p8` file into JSON, make sure to replace physical line breaks with `\n` so it remains a valid, single-line JSON string.
:::
</TabItem>
</Tabs>
---
### How Multi-Scrobbler handles Apple Music quirks
#### Timestamp Estimation
The Apple Music API **does not provide timestamps** for when tracks were played. Multi-Scrobbler estimates play times by taking the current time and subtracting track durations backwards:
* The most recent track is assumed to have finished playing **now**.
* Each older track is estimated to have played `duration` seconds before the previous one.
*For the most accurate scrobble timestamps, it is highly recommended to keep the polling interval (`APPLEMUSIC_INTERVAL`) low (the default is 60 seconds).*
#### Duplicate Play Recovery
The Apple Music history API strictly deduplicates tracks. Instead of showing the same song multiple times in a row, it simply bumps the re-played track back to the #1 spot and removes the older entry.
For example, if you listen to **Song A** → **Song B**, your history is `[Song B, Song A]`. If you then listen to **Song A** again, the API returns `[Song A, Song B]` (Song A was bumped to the top).
If Multi-Scrobbler polled when the top track was Song A, and polls again after it's bumped back to the top, the top track appears functionally unchanged. This usually confuses standard scrobblers into ignoring the update entirely. By default, MS correctly detects this "top-rebound" pattern, extracts the intermediate tracks that were squeezed in (Song B), and scrobbles Song A again as a re-listen.
*Note: Because of this strict deduplication, **"pure loops"** (listening to Song A continuously on repeat without playing anything else in between) cannot be tracked. The history list never changes shape, so MS has no way of knowing it was played again.*
If you ever experience false positives (tracks being scrobbled that you didn't actually re-listen to), you can disable this recovery behavior with `"recoverUnchangedTopHistory": false` in your config options or by setting the `APPLEMUSIC_RECOVER_UNCHANGED_TOP_HISTORY=false` env var.
#### Album name normalization
The Apple Music API appends ` - EP` or ` - Single` to the album name for EPs and singles. These suffixes are *not* part of the official names of the album and can cause issues when trying to scrobble or match metadata.
By default, Multi-Scrobbler strips these suffixes out so only the official name is used.
If you do not want this behavior it can be disabled with ENV `APPLEMUSIC_NORMALIZE_ALBUM=false` or File/AIO option `"normalizeAlbum": false`.
<DetailsAdmo type="tip" summary="Stripping Suffix with User Stage">
If you still want this stripping functionality but with more control over how it is applied you can use a [User Stage](/configuration/transforms/user).
Here is an example of creating a User Stage to only strip suffixes for a specific scrobble client:
<details>
<summary>Example</summary>
```json5 title="config.json"
{
// ...
"transformers": [
{
"type": "user",
"name": "AppleAlbumStip",
"defaults": {
"album": [
"/ - (EP|Single)$/i"
]
}
}
]
}
```
```json5 title="applemusic.json"
{
{
"id": "myAppleMusic",
"name": "My Apple Music",
"data": {
// ...
},
// highlight-start
"options": {
"normalizeAlbum": false
}
// highlight-end
}
}
```
```json5 title="lastfm.json"
[
{
"name": "Foxx LFM Client",
"id": "myLastFmClient",
"enable": true,
"configureAs": "client",
"data": {
"apiKey": "a89cba1569901a0671d5a9875fed4be1",
"secret": "ec42e09d5ae0ee0f0816ca151008412a",
"redirectUri": "http://localhost:9078/lastfm/callback"
},
// highlight-start
"options": {
"playTransform": {
"preCompare": [
{
"type": "user",
"name": "AppleAlbumStip",
}
]
}
}
// highlight-end
}
]
```
</details>
</DetailsAdmo>
## Configuration Reference
<Config config="AppleMusicSourceConfig" fileContent={AppleMusicConfig} name="applemusic">
| Environment Variable | Required? | Default | Description |
| ------------------------------------------ | --------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `APPLEMUSIC_ID` | Yes | | A unique ID for this source. |
| `APPLEMUSIC_MEDIA_USER_TOKEN` | Yes | | The media-user-token extracted from the browser. |
| `APPLEMUSIC_KEY_ID` | No | | Key ID from your MusicKit key. |
| `APPLEMUSIC_TEAM_ID` | No | | Team ID from your Apple Developer account. |
| `APPLEMUSIC_KEY_P8` | No | | The contents of your MusicKit `.p8` private key file. |
| `APPLEMUSIC_TOKEN` | No | | The authentication JWT extracted from the browser. |
| `APPLEMUSIC_INTERVAL` | No | `60` | Polling interval in seconds. |
| `APPLEMUSIC_ORIGIN_HEADER` | No | | Origin header for API requests, e.g. `https://music.apple.com`. Required when using a browser token. |
| `APPLEMUSIC_RECOVER_UNCHANGED_TOP_HISTORY` | No | `true` | Apple Music deduplicates its history by bumping re-played tracks to the top. If you listen to Song A → Song B → Song A, the history changes from `[A]` to `[A, B]` (bumping A). MS detects this "top-rebound" to recover the intermediate play (B) and the re-listen (A). Disable this only if you see incorrect duplicate scrobbles. |
| `APPLEMUSIC_NORMALIZE_ALBUM` | No | `true` | Strips extraneous `- EP` and `- Single` suffixes from album names |
| `APPLEMUSIC_NAME` | No | | A vanity name different than the ID. |
</Config>
@@ -15,6 +15,7 @@ A **Source** is a data source that contains information about tracks you are pla
| Name | Networking | Scrobble SOT | [Multi Device/User](#multi-deviceuser) | [Scrobble Destination](#limiting-scrobble-destination) | [Thresholds](#scrobble-thresholds) | [Should Scrobble](#should-scrobble-behavior) |
| :---------------------------------------------------------------------- | :------------------------------------------------------- | :-------------------------------------------------- | :------------------------------------- | :----------------------------------------------------- | :--------------------------------- | :------------------------------------------- |
| [Apple Music](/configuration/sources/applemusic) | [Active](./?sourceComm=active#by-communication-method) | [History](./?sot=history#by-data-source-of-truth) | ❌ | ✅ | ❌ | ❌ |
| [Azuracast](/configuration/sources/azuracast) | [Active](./?sourceComm=active#by-communication-method) | [Activity](./?sot=activity#by-data-source-of-truth) | ❌ | ✅ | ✅ | ✅ |
| [Deezer](/configuration/sources/deezer) | [Active](./?sourceComm=active#by-communication-method) | [History](./?sot=history#by-data-source-of-truth) | ✅ | ✅ | ❌ | ❌ |
| [Google Cast (Chromecast)](/configuration/sources/google-cast) | [Active](./?sourceComm=active#by-communication-method) | [Activity](./?sot=activity#by-data-source-of-truth) | ✅ | ✅ | ✅ | ❌ |
+1
View File
@@ -13,6 +13,7 @@ sidebar_position: 1
A dockerized app that monitors your music listening activity from *everywhere* and scrobbles it *anywhere*.
* Supports monitoring activity from many [**Sources**](/configuration/sources)
* [Apple Music](/configuration/sources/applemusic)
* [Azuracast](/configuration/sources/azuracast)
* [Deezer](/configuration/sources/deezer)
* [Google Cast (Chromecast)](/configuration/sources/google-cast)
+46 -1
View File
@@ -42,7 +42,7 @@ Cross-platform images are built for x86 (Intel/AMD) and ARM64 (IE Raspberry Pi)
<Tabs groupId="dockerSetting" queryString>
<TabItem value="storage" label="Storage">
You **should** bind a host directory into the container for storing configurations and credentials. Otherwise, these will be lost when the container is updated.
You **should** bind a host directory into the container for storing configurations, credentials, database, and other files. Otherwise, these will be lost when the container is updated.
```yaml title="docker-compose.yml"
services:
@@ -54,6 +54,24 @@ services:
// highlight-end
```
Optionally, use a separate directory for data (database, cache, etc...) by mounting an additional directory and setting the `DATA_DIR` ENV to the directory that should be used inside the container:
```yaml title="docker-compose.yml"
services:
multi-scrobbler:
# ...
environment:
# ...
// highlight-start
- DATA_DIR=/msData
// highlight-end
volumes:
- "./config:/config" # used for config files
// highlight-start
- "./my/host/folder:/msData" # used for data files
// highlight-end
```
</TabItem>
<TabItem value="networking" label="Networking">
If you are using a [bridge network](https://www.appsdeveloperblog.com/docker-networking-bridging-host-and-overlay/) (default docker setup) you must map a port to the container in order to access the dashboard and use MS with some sources (Webscrobbler, LFM/LZ Endpoints). The default container port is `9078`.
@@ -231,6 +249,33 @@ npm run docs:install && npm run build
npm run start
```
#### Config And Data Directories
The directories MS uses for **Configuration** (`config.json`, `jellyfin.json`, etc...) and **Data** (database, cache, etc...) can be specified by environmental variables passed directly to `npm`/`node`, or set in an `.env` file in the working directory.
When parsing these ENVs MS will resolve relative directories but *not* bash expansions like `~` for home. Examples:
* `CONFIG_DIR=/my/absolute/path` => sets directory to use for config files
* `DATA_DIR=./relative/data` => sets directory relative to working directory for data files
If you do not set these environmental variables then MS will attempt to use OS-standardized directories:
<Tabs>
<TabItem value="linux" label="Linux">
* Config => `$XDG_CONFIG_HOME/multi-scrobbler` or `~/.config/multi-scrobbler`
* Data => `$XDG_DATA_HOME/multi-scrobbler` or `~/.local/share/multi-scrobbler`
</TabItem>
<TabItem value="mac" label="MacOS">
* Config => `~/Library/Preferences/multi-scrobbler`
* Data => `~/Library/Application Support/multi-scrobbler`
</TabItem>
<TabItem value="win" label="Windows">
* Config => `%APPDATA%\multi-scrobbler\Config`
* Data => `%LOCALAPPDATA%\multi-scrobbler\Data`
</TabItem>
</Tabs>
#### Rollup build error
During building if you encounter an error like: `Your current platform "XXX" and architecture "XXX" combination is not yet supported by the native Rollup build.`
+5 -4
View File
@@ -1,13 +1,14 @@
import 'dotenv/config';
import { defineConfig } from 'drizzle-kit';
import { configDir, projectDir } from './src/backend/common/index.js';
import { getDataDir } from './src/backend/common/index.js';
import * as path from 'path';
import { projectRootDir } from './src/core/Atomic.ts';
export default defineConfig({
schema: path.resolve(projectDir, 'src/backend/common/database/drizzle/schema'),
out: path.resolve(projectDir, 'src/backend/common/database/drizzle/migrations'),
schema: path.resolve(projectRootDir, 'src/backend/common/database/drizzle/schema'),
out: path.resolve(projectRootDir, 'src/backend/common/database/drizzle/migrations'),
dialect: 'sqlite',
dbCredentials: {
url: path.resolve(configDir, 'ms.db'),
url: path.resolve(getDataDir(), 'ms.db'),
},
});
+75 -5
View File
@@ -2,6 +2,7 @@
import storybook from "eslint-plugin-storybook";
import boundaries from 'eslint-plugin-boundaries';
import unusedImports from 'eslint-plugin-unused-imports';
import { importX } from 'eslint-plugin-import-x'
import { defineConfig, globalIgnores } from "eslint/config";
@@ -13,6 +14,7 @@ import tsEslint from 'typescript-eslint';
import arrow from 'eslint-plugin-prefer-arrow-functions';
import hooks from 'eslint-plugin-react-hooks';
import mochaPlugin from 'eslint-plugin-mocha';
import unicorn from 'eslint-plugin-unicorn';
const defaultRules = {
'no-useless-catch': 'off',
@@ -70,10 +72,66 @@ export default defineConfig([
plugins: {
"prefer-arrow-functions": arrow,
js,
'unused-imports': unusedImports
'unused-imports': unusedImports,
unicorn,
'import-x': importX,
},
rules: {
...defaultRules,
// https://typescript-eslint.io/rules/consistent-type-imports/#comparison-with-importsnotusedasvalues--verbatimmodulesyntax
//
// when using tsconfig compiler verbatimModuleSyntax and EX import {type Foo, type Bar} from 'a';
// true => typescript will *still* import a module if all types are inline
// false => typescript will erase the entire module import
//
// we need to use verbatimModuleSyntax: true for nodejs type stripping compatibility so
// its important that we use top-level type imports so we don't accidentally import server-side modules into frontend
// but can still use types where necessary
//
// this rule *should* do this...it does detect imports that are typed but not explicitly declared
// but its not moving inline -> top-level
"@typescript-eslint/consistent-type-imports": [
"error",
{
prefer: 'type-imports',
fixStyle: "separate-type-imports"
}
],
// however, import/consistent-type-specifier-style from eslint-plugin-import *does* move inline => top-level
// but it does not yet support eslint10 :(
//
// TODO eventually add this once plugin-import supports eslint10
// https://github.com/import-js/eslint-plugin-import
// https://github.com/import-js/eslint-plugin-import/blob/main/docs/rules/consistent-type-specifier-style.md
// -- this enforces that type-only imports are fixed to top-level type imports IE
// import {type Foo, type Bar} from 'a'; => import type { Foo, Bar} from 'a';
//
// USE eslint-plugin-import-x since its more actively developed?
// waiting on the status of this for supporting mixed inline types
// https://github.com/un-ts/eslint-plugin-import-x/issues/501
// "import-x/consistent-type-specifier-style": [
// "error", "prefer-top-level"
// ],
"unicorn/prefer-then-catch": "error",
"unicorn/consistent-destructuring": "warn",
"unicorn/consistent-function-scoping": "warn",
"unicorn/consistent-optional-chaining": "error",
"unicorn/no-array-callback-reference": "warn",
"unicorn/no-accidental-bitwise-operator": "warn",
"no-obj-calls": "error",
"unicorn/new-for-builtins": "error",
"unicorn/no-impossible-length-comparison": "error",
"unicorn/no-duplicate-loops": "warn",
"unicorn/no-duplicate-logical-operands": "warn",
"unicorn/no-declarations-before-early-exit": "warn",
"unicorn/prefer-negative-index": "warn",
"unicorn/prefer-import-meta-properties": "error",
"unicorn/prefer-array-from-async": "warn",
"unicorn/prefer-array-flat-map": "warn",
"unicorn/no-useless-else": "warn",
"unicorn/no-unused-array-method-return": "error",
//"unicorn/no-unreadable-object-destructuring": "warn",
"unicorn/prefer-object-destructuring-defaults": "warn"
},
extends: [
tsEslint.configs.recommended,
@@ -82,7 +140,9 @@ export default defineConfig([
languageOptions: {
globals: {
...globals.node,
}
},
ecmaVersion: 'latest',
sourceType: 'module',
},
files: ['src/**/*.ts','src/**/*.tsx']
},
@@ -148,7 +208,9 @@ export default defineConfig([
{ type: 'frontend', mode: 'file', pattern: 'src/client/**/*' },
{ type: 'config', mode: 'file', pattern: 'config/*.example' },
{ type: 'core', mode: 'file', pattern: ['src/core/!(tests)/**','src/core/!(tests)'] },
{ type: 'core-tests', mode: 'file', pattern: ['src/core/tests/**'] },
{ type: 'backend', mode: 'file', pattern: 'src/backend/**/*' },
{ type: 'stories', mode: 'file', pattern: ['src/stories/**/*', '.storybook/**'] },
],
// So it understands TS path aliases when resolving imports
'import/resolver': {
@@ -157,7 +219,7 @@ export default defineConfig([
},
rules: {
'boundaries/element-types': [
'error',
'warn',
{
default: 'disallow', // deny anything not explicitly allowed
rules: [
@@ -171,14 +233,22 @@ export default defineConfig([
},
{
from: 'backend',
allow: ['backend', 'core', 'config'], // backend can use itself + core
allow: ['backend', 'core', 'config', 'core-tests'], // backend can use itself + core + tests
},
{
from: 'stories',
allow: ['stories', 'core', 'frontend', 'core-tests'], // backend can use itself + core
},
{
from: 'core-tests',
allow: ['core', 'core-tests'], // backend can use itself + core
},
],
},
],
// Optional: flag any file under src/ that doesn't match one of the
// three element patterns above (catches stray/misplaced files)
'boundaries/no-unknown': 'error'
'boundaries/no-unknown': 'warn'
},
}
]);
+548 -188
View File
File diff suppressed because it is too large Load Diff
+9 -3
View File
@@ -25,7 +25,8 @@
"cliff": "git-cliff --unreleased",
"storybook": "storybook dev -p 6006",
"build-storybook": "storybook build",
"atproto:lex:tealfm": "npm exec -- lex-cli pull -c src/backend/common/vendor/teal/lex.config.ts && npm exec -- lex-cli generate -c src/backend/common/vendor/teal/lex.config.ts"
"atproto:lex:tealfm": "npm exec -- lex-cli pull -c src/backend/common/vendor/teal/lex.config.ts && npm exec -- lex-cli generate -c src/backend/common/vendor/teal/lex.config.ts",
"frontend:analyze": "npx esbuild src/client/index-next.tsx --bundle --platform=node --metafile=meta.json --outfile=/dev/null"
},
"exports": {
".": {
@@ -81,7 +82,7 @@
"avahi-browse": "^1.1.4",
"better-sse": "^0.8.0",
"body-parser": "^2.2.2",
"cacheable": "^1.10.4",
"cacheable": "^2.5.0",
"castv2": "^0.1.10",
"clone": "^2.1.2",
"common-tags": "^1.8.2",
@@ -93,6 +94,7 @@
"discord.js": "^14.26.0",
"dotenv": "^10.0.0",
"drizzle-orm": "^1.0.0-rc.2-c5a84d1",
"env-paths": "^4.0.0",
"express": "^5.2.1",
"express-session": "^1.19.0",
"fast-equals": "^6.0.0",
@@ -109,7 +111,7 @@
"json-diff-ts": "^5.0.0-alpha.2",
"json5": "^2.2.3",
"jsondiffpatch": "^0.7.3",
"keyv": "^5.5.0",
"keyv": "^5.6.0",
"kodi-api": "^0.2.1",
"lastfm-ts-api": "^2.6.0",
"merge-error-cause": "^5.0.2",
@@ -118,6 +120,7 @@
"musicbrainz-api": "^0.27.0",
"nanoid": "^3.3.1",
"neotraverse": "^0.6.18",
"node-musickit-api": "^3.1.0-next.5",
"node-object-hash": "^3.1.1",
"normalize-url": "^8.0.1",
"ntfy": "^1.15.2",
@@ -210,10 +213,12 @@
"eslint": "^10.6.0",
"eslint-import-resolver-typescript": "^4.4.5",
"eslint-plugin-boundaries": "^7.0.2",
"eslint-plugin-import-x": "^4.17.1",
"eslint-plugin-mocha": "^11.3.0",
"eslint-plugin-prefer-arrow-functions": "^3.2.4",
"eslint-plugin-react-hooks": "^7.1.1",
"eslint-plugin-storybook": "^10.3.6",
"eslint-plugin-unicorn": "^72.0.0",
"eslint-plugin-unused-imports": "^4.4.1",
"git-cliff": "^2.12.0",
"globals": "^17.7.0",
@@ -243,6 +248,7 @@
"ts-json-schema-generator": "^2.3.0",
"typescript": "^6.0.3",
"typescript-eslint": "^8.59.3",
"usehooks-ts": "^3.1.1",
"vite": "^8.0.12",
"with-local-tmp-dir": "^7.0.1"
},
+20 -20
View File
@@ -2,11 +2,11 @@ import { childLogger, type Logger } from "@foxxmd/logging";
import {
cacheFunctions,
} from "@foxxmd/regex-buddy-core";
import EventEmitter from "events";
import { type ComponentType, type LifecycleInput, type LifecycleStep, type PlayData, type PlayObject } from "../../core/Atomic.ts";
import type EventEmitter from "events";
import type {ComponentType, LifecycleInput, LifecycleStep, PlayData, PlayObject} from "../../core/Atomic.ts";
import { buildTrackString } from "../../core/StringUtils.ts";
import { type CommonClientConfig } from "./infrastructure/config/client/index.ts";
import { type CommonSourceConfig } from "./infrastructure/config/source/index.ts";
import type {CommonClientConfig} from "./infrastructure/config/client/index.ts";
import type {CommonSourceConfig} from "./infrastructure/config/source/index.ts";
import { mergeSimpleError, SimpleError, SkipTransformStageError, StagePrerequisiteError, StageTransformError, TransformRulesError } from "./errors/MSErrors.ts";
import {
FLOW_CONTROL_TERM,
@@ -16,29 +16,29 @@ import {
type TransformHook
} from "../../core/Transform.ts";
import AbstractInitializable from "./AbstractInitializable.ts";
import TransformerManager from "./transforms/TransformerManager.ts";
import type TransformerManager from "./transforms/TransformerManager.ts";
import { getRoot } from "../ioc.ts";
import { nanoid } from "nanoid";
import { isDebugMode } from "../utils.ts";
import { findCauseByFunc, findCauseByReference } from "../utils/ErrorUtils.ts";
import { hashObject, parseArrayFromMaybeString } from "../utils/StringUtils.ts";
import { playContentInvariantTransform } from "../utils/PlayComparisonUtils.ts";
import { MSCache } from "./Cache.ts";
import type { MSCache } from "./Cache.ts";
import { diffObjects, diffObjectsConsoleOutput, patchObject } from "../../core/DataUtils.ts";
import clone from "clone";
import { loggerNoop } from "./MaybeLogger.ts";
import { objectsEqual } from "../utils/DataUtils.ts";
import { type RetentionOptions } from "./infrastructure/config/database.ts";
import type {RetentionOptions} from "./infrastructure/config/database.ts";
import { getRetentionCompactAfterFromEnv, getRetentionDeleteAfterFromEnv, isCompactableProperty, parseRetentionOptions, parseRetentionOptionsDurations } from "./database/Database.ts";
import { type DbConcrete } from "./database/drizzle/drizzleUtils.ts";
import { type ComponentSelect } from "./database/drizzle/drizzleTypes.ts";
import type {DbConcrete} from "./database/drizzle/drizzleUtils.ts";
import type {ComponentSelect} from "./database/drizzle/drizzleTypes.ts";
import { DrizzlePlayRepository } from "./database/drizzle/repositories/PlayRepository.ts";
import { type ClientType } from "./infrastructure/config/client/clients.ts";
import { type SourceType } from "./infrastructure/config/source/sources.ts";
import type {ClientType} from "../../core/Atomic.ts";
import type {SourceType} from "../../core/Atomic.ts";
import { DrizzleComponentRepository } from "./database/drizzle/repositories/ComponentRepository.ts";
import dayjs from "dayjs";
import { COMPONENT_STATE, type ComponentCommonApi, type ComponentCommonApiJson, type ComponentState, type PlayApiCommonDetailed } from "../../core/Api.ts";
import { type WebhookPayload } from "./infrastructure/config/health/webhooks.ts";
import type {WebhookPayload} from "./infrastructure/config/health/webhooks.ts";
import type { MarkRequired } from "ts-essentials";
import { serializeError } from "serialize-error";
@@ -79,6 +79,10 @@ export default abstract class AbstractComponent extends AbstractInitializable {
};
}
public getUid() {
return this.config?.id ?? this.config?.name ?? this.name;
}
protected postCache(): Promise<void> {
try {
this.buildTransformRules();
@@ -96,18 +100,13 @@ export default abstract class AbstractComponent extends AbstractInitializable {
protected async doBuildDatabase(): Promise<true | string | undefined> {
await super.doBuildDatabase();
let name: string;
if('name' in this) {
name = this.name as string;
}
this.db = await getRoot().items.db();
this.componentRepo = new DrizzleComponentRepository(this.db, {logger: this.logger});
this.dbComponent = await this.componentRepo.findOrInsert({
mode: this.componentType,
type: this.type,
uid: this.config.id ?? this.config.name ?? name,
name: this.config.name ?? name
uid: this.getUid(),
name: this.config?.name ?? this.name
});
this.componentId = this.dbComponent.id;
return true;
@@ -473,6 +472,7 @@ export default abstract class AbstractComponent extends AbstractInitializable {
step.flowReason = skipMsg;
logger.debug(merged, skipMsg);
return [step, playTruth];
} else {
step.flowResult = onFailure;
let reason: string;
@@ -520,7 +520,7 @@ export default abstract class AbstractComponent extends AbstractInitializable {
delete newTransformedPlay.meta.lifecycleInputs;
}
return [step, newTransformedPlay];
return [step, newTransformedPlay ?? playTruth];
}
public abstract getRunningState(): ComponentState
+2 -2
View File
@@ -1,8 +1,8 @@
import { type Logger } from "@foxxmd/logging";
import type { Logger } from "@foxxmd/logging";
import {truncateStringToLength } from "../../core/StringUtils.ts";
import { hasNodeNetworkException } from "./errors/NodeErrors.ts";
import { hasUpstreamError } from "./errors/UpstreamError.ts";
import { type WebhookPayload } from "./infrastructure/config/health/webhooks.ts";
import type {WebhookPayload} from "./infrastructure/config/health/webhooks.ts";
import { AuthCheckError, BuildDataError, ConnectionCheckError, ParseCacheError, PostInitError, StageError } from "./errors/MSErrors.ts";
import { messageWithCausesTruncatedDefault } from "../../core/ErrorUtils.ts";
import { spawn } from 'abort-controller-x';
+9 -8
View File
@@ -10,7 +10,7 @@ import timezone from 'dayjs/plugin/timezone.js';
import utc from 'dayjs/plugin/utc.js';
import clone from 'clone';
import { childLogger, type Logger } from '@foxxmd/logging';
import { projectDir } from './index.ts';
import { getConfigDir } from './index.ts';
import path from 'path';
import { cacheFunctions } from "@foxxmd/regex-buddy-core";
import { fileOrDirectoryIsWriteable } from '../utils/FSUtils.ts';
@@ -18,8 +18,8 @@ import { asCacheConfig, type CacheAuthProvider, type CacheConfig, type CacheConf
import { Typeson } from 'typeson';
import { builtin } from 'typeson-registry';
import { loggerNoop } from './MaybeLogger.ts';
const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
import prom, { Gauge } from 'prom-client';
import type { Gauge } from 'prom-client';
import prom from 'prom-client';
import { nonEmptyStringOrDefault } from '../../core/StringUtils.ts';
dayjs.extend(utc)
@@ -276,11 +276,12 @@ export class MSCache {
throw e;
}
}
const confDir = getConfigDir();
if (config.provider === 'file') {
logger.debug(`Building file cache from ${path.join(config.connection ?? configDir, `${namespace}.cache`)}`);
logger.debug(`Building file cache from ${path.join(config.connection ?? confDir, `${namespace}.cache`)}`);
try {
const [keyvFile] = await initFileCache({ ...config, cacheDir: config.connection ?? configDir, cacheId: `${namespace}.cache` }, {ttl: config.ttl}, logger);
const [keyvFile] = await initFileCache({ ...config, cacheDir: config.connection ?? confDir, cacheId: `${namespace}.cache` }, {ttl: config.ttl}, logger);
return keyvFile;
} catch (e) {
throw e;
@@ -537,7 +538,7 @@ export const parseUserConfig = (config: CacheConfigUser = {}, parentLogger: Logg
// } = {},
scrobble: {
provider: sProvider = (process.env.CACHE_SCROBBLE as (CacheScrobbleProvider | undefined) ?? 'file'),
connection = (process.env.CACHE_SCROBBLE_CONN ?? configDir),
connection = (process.env.CACHE_SCROBBLE_CONN ?? getConfigDir()),
...restScrobble
} = {},
auth: {
@@ -562,7 +563,7 @@ export const parseUserConfig = (config: CacheConfigUser = {}, parentLogger: Logg
if(authProvider === 'valkey') {
if(valkey === undefined) {
logger.warn(`Auth Provider set to 'valkey' but not valkey connection string was not provided, falling back to file.`);
authConn = configDir;
authConn = getConfigDir();
authProvider = 'file';
} else {
authConn = valkey;
@@ -571,7 +572,7 @@ export const parseUserConfig = (config: CacheConfigUser = {}, parentLogger: Logg
if(authProvider !== 'file') {
logger.warn(`Unsupported provider given for auth: ${authProvider}`);
}
authConn = configDir;
authConn = getConfigDir();
authProvider = 'file';
}
+1 -1
View File
@@ -1,4 +1,4 @@
import { type Logger } from "@foxxmd/logging";
import type { Logger } from "@foxxmd/logging";
export class MaybeLogger {
+4 -4
View File
@@ -1,10 +1,10 @@
import { configDir } from '../index.ts';
import { getDataDir } from '../index.ts';
import * as path from 'path';
import { childLogger, type Logger } from '@foxxmd/logging';
import { loggerNoop } from '../MaybeLogger.ts';
import { COMPACTABLE, type CompactableProperty, DEFAULT_RETENTION_COMPACT_AFTER, DEFAULT_RETENTION_DELETE_AFTER, type RetentionConfigValue, type RetentionOption, type RetentionValue, type RetentionValueUnparsed } from '../infrastructure/config/database.ts';
import { type DurationValue } from '../infrastructure/Atomic.ts';
import { type Duration } from 'dayjs/plugin/duration.js';
import type {DurationValue} from '../infrastructure/Atomic.ts';
import type {Duration} from 'dayjs/plugin/duration.js';
import dayjs from 'dayjs';
import { parseDurationFromDurationValue } from '../../utils/TimeUtils.ts';
import assert from 'node:assert';
@@ -19,7 +19,7 @@ export const getDbPath = (name: string = 'ms', workingDirectory?: string): strin
if (isMemoryDb(name)) {
return MEMORY_DB_NAME;
}
return path.resolve(workingDirectory ?? configDir, `${name}.db`);
return path.resolve(workingDirectory ?? getDataDir(), `${name}.db`);
}
export const getDbBackupPath = (dbPath: string, suffix?: string): string => {
@@ -1,9 +1,9 @@
import type { SqliteDatabase, Migration } from 'sqlite-up';
import { type MigrateBaseContext } from '../appMigrator.ts';
import type {MigrateBaseContext} from '../appMigrator.ts';
import { plays as drizzlePlays } from '../drizzle/schema/schema.ts';
import clone from 'clone';
import { eq } from 'drizzle-orm';
import { type PlayLifecycle, type PlayObject } from '../../../../core/Atomic.ts';
import type {PlayLifecycle, PlayObject} from '../../../../core/Atomic.ts';
export const up: Migration<MigrateBaseContext>['up'] = async (db: SqliteDatabase, ctx: MigrateBaseContext): Promise<void> => {
+5 -5
View File
@@ -1,10 +1,10 @@
import { type DbConcrete } from "./drizzle/drizzleUtils.ts";
import type {DbConcrete} from "./drizzle/drizzleUtils.ts";
import { loggerNoop } from "../MaybeLogger.ts";
import * as path from 'path';
import { childLogger, type Logger } from "@foxxmd/logging";
import { projectDir } from "../index.ts";
import { Migrator } from 'sqlite-up';
import { type MigrationStatus } from "../infrastructure/Atomic.ts";
import type {MigrationStatus} from "../infrastructure/Atomic.ts";
import { projectRootDir } from "../../../core/Atomic.ts";
export interface MigrateBaseContext {
db: DbConcrete
@@ -16,7 +16,7 @@ const migrationsTable = '__app_migrations';
export const getAppMigrationStatus = async (db: DbConcrete, opts: {logger?: Logger, migrationsAppFolder?: string} = {}): Promise<MigrationStatus> => {
const {
logger: parentLogger = loggerNoop,
migrationsAppFolder = path.resolve(projectDir, 'src/backend/common/database/appMigrations')
migrationsAppFolder = path.resolve(projectRootDir, 'src/backend/common/database/appMigrations')
} = opts;
const logger = childLogger(parentLogger, 'App Migrations');
@@ -40,7 +40,7 @@ export const getAppMigrationStatus = async (db: DbConcrete, opts: {logger?: Logg
export const migrateApp = async (db: DbConcrete, opts: {logger?: Logger, migrationsAppFolder?: string} = {}): Promise<string[]> => {
const {
logger: parentLogger = loggerNoop,
migrationsAppFolder = path.resolve(projectDir, 'src/backend/common/database/appMigrations')
migrationsAppFolder = path.resolve(projectRootDir, 'src/backend/common/database/appMigrations')
} = opts;
const logger = childLogger(parentLogger, ['App', 'Migrations']);
@@ -1,6 +1,6 @@
import { type DBQueryConfig, type DBQueryConfigWith, type KnownKeysOnly, type RelationFieldsFilterInternals, type BuildQueryResult, type RelationsFilter } from "drizzle-orm";
import { components, componentMigrations, playInputs, plays, queueStates, relations, playsHistorical } from "./schema/schema.ts";
import {type TSchema, type TableName } from "./schema/schema.ts";
import type {DBQueryConfig, DBQueryConfigWith, KnownKeysOnly, RelationFieldsFilterInternals, BuildQueryResult, RelationsFilter} from "drizzle-orm";
import type { components, componentMigrations, playInputs, plays, queueStates, relations, playsHistorical } from "./schema/schema.ts";
import type {TSchema, TableName} from "./schema/schema.ts";
export type ComponentNew = typeof components.$inferInsert;
@@ -1,6 +1,6 @@
import { drizzle } from 'drizzle-orm/node-sqlite';
import { migrate } from 'drizzle-orm/node-sqlite/migrator';
import { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core";
import type { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core";
import { sql as dsl, type Logger as DrizzleLogger } from 'drizzle-orm';
import * as fs from 'fs/promises';
import * as path from 'path';
@@ -8,16 +8,16 @@ import { backupDb, getDbBackupPath, MEMORY_DB_NAME } from '../Database.ts';
import { fileExists, fileOrDirectoryIsWriteable } from '../../../utils/FSUtils.ts';
import { childLogger, type Logger, type LogLevel } from '@foxxmd/logging';
import { loggerNoop } from '../../MaybeLogger.ts';
import { projectDir } from '../../index.ts';
import { relations } from './schema/schema.ts';
import { addToContext, executeQuery } from './logContext.ts';
import { migrateApp, getAppMigrationStatus } from '../appMigrator.ts';
import { type MigrationStatus } from '../../infrastructure/Atomic.ts';
import type {MigrationStatus} from '../../infrastructure/Atomic.ts';
import { projectRootDir } from '../../../../core/Atomic.ts';
export async function getDbMigrationStatus(dbVal: string | DbConcrete, opts: {logger?: Logger, migrationsFolder?: string} = {}): Promise<MigrationStatus> {
const {
logger: parentLogger = loggerNoop,
migrationsFolder = path.resolve(projectDir, 'src/backend/common/database/drizzle/migrations')
migrationsFolder = path.resolve(projectRootDir, 'src/backend/common/database/drizzle/migrations')
} = opts;
const logger = childLogger(parentLogger, 'Migrations');
@@ -91,7 +91,7 @@ export const migrateDb = async (db: DbConcrete, opts: {logger?: Logger, migratio
try {
logger.info('Starting migrations...');
await executeQuery('migrations', async () => migrate(db, { migrationsFolder: migrationsFolder ?? path.resolve(projectDir, 'src/backend/common/database/drizzle/migrations') }), logger, process.env.LOG_MIGRATION === 'true' ? true : 'error');
await executeQuery('migrations', async () => migrate(db, { migrationsFolder: migrationsFolder ?? path.resolve(projectRootDir, 'src/backend/common/database/drizzle/migrations') }), logger, process.env.LOG_MIGRATION === 'true' ? true : 'error');
logger.info('Migrations complete');
} catch (e) {
throw new Error('Failed to migrate database', { cause: e });
@@ -107,7 +107,7 @@ export const migrateDbSync = (db: ReturnType<typeof drizzle>, opts: {logger?: Lo
try {
logger.info('Starting migrations...');
migrate(db, { migrationsFolder: migrationsFolder ?? path.resolve(projectDir, 'src/backend/common/database/drizzle/migrations') });
migrate(db, { migrationsFolder: migrationsFolder ?? path.resolve(projectRootDir, 'src/backend/common/database/drizzle/migrations') });
logger.info('Migrations complete');
} catch (e) {
throw new Error('Failed to migrate database', { cause: e });
@@ -1,8 +1,8 @@
import assert from "node:assert";
import { type PlayHistoricalNew, type PlayHistoricalSelect, type PlayNew, type PlaySelect, type PlaySelectWithQueueStates } from "./drizzleTypes.ts";
import { type PlayInputNew } from "./drizzleTypes.ts";
import { type QueueStateNew } from "./drizzleTypes.ts";
import { type ComponentNew } from "./drizzleTypes.ts";
import type {PlayHistoricalNew, PlayHistoricalSelect, PlayNew, PlaySelect, PlaySelectWithQueueStates} from "./drizzleTypes.ts";
import type {PlayInputNew} from "./drizzleTypes.ts";
import type {QueueStateNew} from "./drizzleTypes.ts";
import type {ComponentNew} from "./drizzleTypes.ts";
import type { MarkOptional } from "ts-essentials";
import { CLIENT_DEAD_QUEUE, type DeadLetterScrobble, type ErrorLike, type PlayObject } from "../../../../core/Atomic.ts";
import dayjs from "dayjs";
@@ -1,4 +1,4 @@
import { type Logger } from '@foxxmd/logging'
import type { Logger } from '@foxxmd/logging'
import { AsyncLocalStorage } from 'async_hooks'
// based on https://numeric.substack.com/p/upgrading-drizzleorm-logging-with
@@ -1,13 +1,13 @@
import { childLogger, type Logger } from "@foxxmd/logging";
import { type DbConcrete } from "../drizzleUtils.ts";
import { type Dayjs } from "dayjs";
import type { DbConcrete } from "../drizzleUtils.ts";
import type { Dayjs } from "dayjs";
import { type RelationsFieldFilter, eq, inArray } from "drizzle-orm";
import { loggerNoop } from "../../../MaybeLogger.ts";
import { capitalize } from "../../../../../core/StringUtils.ts";
import { getConfigByTableName, type TableName } from "../schema/schema.ts";
import assert from 'node:assert';
import { Cacheable } from "cacheable";
import { type DateLike } from "../../../../../core/Atomic.ts";
import type { Cacheable } from "cacheable";
import type { DateLike } from "../../../../../core/Atomic.ts";
import type { CompareDateBetween, CompareDateSingle } from "../../../../../core/Api.ts";
export interface DrizzleRepositoryOpts {
@@ -1,9 +1,9 @@
import { DrizzleBaseRepository, type DrizzleRepositoryOpts } from "./BaseRepository.ts";
import { type DbConcrete } from "../drizzleUtils.ts";
import { type ComponentSelect, type FindWhere } from "../drizzleTypes.ts";
import type {DbConcrete} from "../drizzleUtils.ts";
import type {ComponentSelect, FindWhere} from "../drizzleTypes.ts";
import { components } from "../schema/schema.ts";
import { generateComponentEntity } from "../entityUtils.ts";
import { type ComponentType } from "../../../../../core/Atomic.ts";
import type {ComponentType} from "../../../../../core/Atomic.ts";
export class DrizzleComponentRepository extends DrizzleBaseRepository<'components'> {
@@ -2,17 +2,17 @@ import { type DbConcrete, runTransaction } from "../drizzleUtils.ts";
import { type PlayObject, TA_DEFAULT_ACCURACY, type TemporalAccuracy } from "../../../../../core/Atomic.ts";
import { generatePlayEntity, hydratePlaySelect, type PlayHydateOptions, type PlayHistoricalEntityOpts } from "../entityUtils.ts";
import { plays, playsHistorical } from "../schema/schema.ts";
import { type FindWhere, type FindMany, type WhereClause, type PlayHistoricalSelect, type PlayHistoricalNew } from "../drizzleTypes.ts";;
import type {FindWhere, FindMany, WhereClause, PlayHistoricalSelect, PlayHistoricalNew} from "../drizzleTypes.ts";;
import type { MarkOptional } from "ts-essentials";
import { removeUndefinedKeys } from '../../../../../core/DataUtils.ts';
import { type Dayjs } from "dayjs";
import type {Dayjs} from "dayjs";
import { inArray, sql } from "drizzle-orm";
import { buildDateCompare, type CompareDateOp, type ComponentConstrainedRepoOpts, DrizzleBaseRepository, type DrizzleRepositoryOpts } from "./BaseRepository.ts";
import { type PaginatedResponse } from "../../../../../core/Api.ts";
import type {PaginatedResponse} from "../../../../../core/Api.ts";
import { hashObject } from "../../../../utils/StringUtils.ts";
import { playContentBasicInvariantTransform, playMbidIdentifier } from "../../../../utils/PlayComparisonUtils.ts";
import { comparePlayTemporally, getTemporalAccuracyCloseVal, hasAcceptableTemporalAccuracy } from "../../../../utils/TimeUtils.ts";
import { type SourceType } from "../../../infrastructure/config/source/sources.ts";
import type {SourceType} from "../../../../../core/Atomic.ts";
import { getTemporallyCloseDateCompareOp } from "./PlayRepository.ts";
// https://github.com/drizzle-team/drizzle-orm/issues/695 may be useful for typing models with relations?
@@ -10,14 +10,14 @@ import { playContentBasicInvariantTransform, playMbidIdentifier } from "../../..
import { hashObject } from "../../../../utils/StringUtils.ts";
import { comparePlayTemporally, getTemporalAccuracyCloseVal, hasAcceptableTemporalAccuracy } from "../../../../utils/TimeUtils.ts";
import { type CompactableProperty, type RetentionOptions, retentionPlayTypes } from "../../../infrastructure/config/database.ts";
import { type SourceType } from "../../../infrastructure/config/source/sources.ts";
import { type FindMany, type FindWhere, type FindWith, type PlayInputNew, type PlayNew, type PlaySelect, type PlaySelectWithQueueStates, type PlayWith, type QueueStateSelect, type WhereClause } from "../drizzleTypes.ts";
import type {SourceType} from "../../../../../core/Atomic.ts";
import type {FindMany, FindWhere, FindWith, PlayInputNew, PlayNew, PlaySelect, PlaySelectWithQueueStates, PlayWith, QueueStateSelect, WhereClause} from "../drizzleTypes.ts";
import { type DbConcrete, runTransaction } from "../drizzleUtils.ts";
import { generateInputEntity, generatePlayEntity, hydratePlaySelect, type PlayEntityOpts, type PlayHydateOptions } from "../entityUtils.ts";
import { playInputs, plays, relations } from "../schema/schema.ts";
import { buildDateCompare, type CompareDateOp, type ComponentConstrainedRepoOpts, DrizzleBaseRepository, type DrizzleRepositoryOpts } from "./BaseRepository.ts";
import { type PaginatedResponse } from "../../../../../core/Api.ts";
import { type PaginatedQueryResponse } from "../../../../../core/Api.ts";
import type {PaginatedResponse} from "../../../../../core/Api.ts";
import type {PaginatedQueryResponse} from "../../../../../core/Api.ts";
;
// https://github.com/drizzle-team/drizzle-orm/issues/695 may be useful for typing models with relations?
@@ -1,7 +1,7 @@
import { eq, and, lte, inArray } from "drizzle-orm";
import { DrizzleBaseRepository, type DrizzleRepositoryOpts } from "./BaseRepository.ts";
import { type DbConcrete } from "../drizzleUtils.ts";
import { type QueueStateSelect } from "../drizzleTypes.ts";
import type {DbConcrete} from "../drizzleUtils.ts";
import type {QueueStateSelect} from "../drizzleTypes.ts";
import { queueStates } from "../schema/schema.ts";
import { CLIENT_DEAD_QUEUE } from "../../../../../core/Atomic.ts";
export class DrizzleQueueRepository extends DrizzleBaseRepository<'queueStates'> {
@@ -1,10 +1,10 @@
import { integer, sqliteTable, text, index, uniqueIndex, customType, type AnySQLiteColumn } from "drizzle-orm/sqlite-core";
import { defineRelations } from 'drizzle-orm';
import dayjs, { type Dayjs } from "dayjs";
import { type ErrorLike, type PlayObject } from "../../../../../core/Atomic.ts";
import { COMPONENT_TYPE_CLIENT, COMPONENT_TYPE_SOURCE, type ErrorLike, type PlayObject } from "../../../../../core/Atomic.ts";
import { asPlayCheap } from "../../../../../core/PlayMarshalUtils.ts";
import { type ExternalMetadataTerm, type PlayTransformPartsConfig, type SearchAndReplaceTerm } from "../../../../../core/Transform.ts";
import { type JobRangeCount, type JobRangeTime } from "../../../infrastructure/Job.ts";
import type {ExternalMetadataTerm, PlayTransformPartsConfig, SearchAndReplaceTerm} from "../../../../../core/Transform.ts";
import type {JobRangeCount, JobRangeTime} from "../../../infrastructure/Job.ts";
import { serializeError, deserializeError } from "serialize-error";
import { generatePlayUid } from "../../../../../core/StringUtils.ts";
@@ -165,7 +165,7 @@ export const components = sqliteTable("components", {
id: integer({ mode: 'number' }).primaryKey(),
// user-provided id
uid: text({ length: 200 }).notNull(),
mode: text({enum: ['source','client']}).notNull(),
mode: text({enum: [COMPONENT_TYPE_SOURCE,COMPONENT_TYPE_CLIENT]}).notNull(),
// spotify, lastfm, etc...
type: text({length: 50}).notNull(),
// vanity display name
+1 -1
View File
@@ -1,4 +1,4 @@
import { type ResponseError } from "superagent";
import type {ResponseError} from "superagent";
export const isSuperAgentResponseError = (e: any): e is ResponseError => {
return typeof e === 'object'
+1 -1
View File
@@ -3,7 +3,7 @@ import mergeErrorCause from 'merge-error-cause';
import { findCauseByFunc, isAbortReasonErrorLike } from "../../utils/ErrorUtils.ts";
import { UpstreamError, type UpstreamErrorOptions } from "./UpstreamError.ts";
import {addKnownErrorConstructor, serializeError} from 'serialize-error';
import { type LifecycleInput } from "../../../core/Atomic.ts";
import type {LifecycleInput} from "../../../core/Atomic.ts";
export abstract class NamedError extends Error {
public abstract name: string;
+1 -1
View File
@@ -1,4 +1,4 @@
import { Response } from 'superagent';
import type { Response } from 'superagent';
import { findCauseByFunc } from "../../utils/ErrorUtils.ts";
import { addKnownErrorConstructor } from 'serialize-error';
+25 -5
View File
@@ -1,8 +1,28 @@
import * as path from 'path';
//import {fileURLToPath} from "url";
import envPaths from 'env-paths';
//const __filename = fileURLToPath(import.meta.url);
//const __dirname = path.dirname(__filename);
const osPaths = envPaths('multi-scrobbler', {suffix: ''});
export const projectDir = process.cwd(); //path.resolve(__dirname, '../../../');
export const configDir: string = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
export const getConfigDir = (): string => {
let configDirVal: string = process.env.CONFIG_DIR ?? osPaths.config;
// this shouldn't happen...
// but if it does we need to have some known path fallback so things don't explode
if(configDirVal === undefined) {
configDirVal = getPathFromCWD('./config'); // backwards compatibility
}
// resolve from relative directory, if one was used
return path.resolve(configDirVal);
}
export const getDataDir = (): string => {
let dataDirVal: string = process.env.DATA_DIR ?? osPaths.data;
// this shouldn't happen...
// but if it does we need to have some known path fallback so things don't explode
if(dataDirVal === undefined) {
dataDirVal = getPathFromCWD('./config'); // defaulting to same directory for backwards compatibility
}
// resolve from relative directory, if one was used
return path.resolve(dataDirVal);
}
export const getPathFromCWD = (...relativePaths: string[]) => path.resolve(process.cwd(), ...relativePaths);
+10 -28
View File
@@ -1,24 +1,21 @@
import { type Logger, type LogDataPretty } from '@foxxmd/logging';
import type { Logger, LogDataPretty } from '@foxxmd/logging';
import type { Dayjs, ManipulateType } from "dayjs";
import { type Request, type Response } from "express";
import type { Request, Response } from "express";
import type { NextFunction, ParamsDictionary, Query } from "express-serve-static-core";
import { FixedSizeList } from 'fixed-size-list';
import { type DeviceId, type ErrorLike, isPlayObject, type PlayMeta, type PlayObject, type PlayObjectMinimal, type PlayPlatformId, type PlayUserId, type UnixTimestamp } from "../../../core/Atomic.ts";
import TupleMap from "../TupleMap.ts";
import { MusicBrainzApi } from 'musicbrainz-api';
import type { SourceType } from './config/source/sources.ts';
import { type ClientType, clientTypes } from './config/client/clients.ts';
import type { FixedSizeList } from 'fixed-size-list';
import { isPlayObject } from "../../../core/Atomic.ts";
import type {DeviceId, ErrorLike, PlayMeta, PlayObject, PlayObjectMinimal, PlayPlatformId, PlayUserId, UnixTimestamp} from "../../../core/Atomic.ts";
import type TupleMap from "../TupleMap.ts";
import type { MusicBrainzApi } from 'musicbrainz-api';
import type { ReportedPlayerStatus, SourceType } from "../../../core/Atomic.ts";
import type { ClientType } from "../../../core/Atomic.ts";
import assert from 'assert';
export interface LeveledLogData extends LogDataPretty {
levelLabel: string
}
export const lowGranularitySources: SourceType[] = ['subsonic', 'ytmusic'];
export const isClientType = (data: string): data is ClientType => {
return clientTypes.includes(data as ClientType);
}
export const lowGranularitySources: SourceType[] = ['subsonic', 'ytmusic', 'applemusic'];
export interface ComponentIdentifier {
type: SourceType | ClientType
@@ -55,21 +52,6 @@ export interface InternalConfig {
export type InternalConfigOptional = Omit<InternalConfig, 'logger'>
export type ReportedPlayerStatus = 'playing' | 'stopped' | 'paused' | 'unknown';
export const REPORTED_PLAYER_STATUSES = {
playing: 'playing' as ReportedPlayerStatus,
stopped: 'stopped' as ReportedPlayerStatus,
paused: 'paused' as ReportedPlayerStatus,
unknown: 'unknown' as ReportedPlayerStatus
}
export type CalculatedPlayerStatus = ReportedPlayerStatus | 'stale' | 'orphaned';
export const CALCULATED_PLAYER_STATUSES = {
...REPORTED_PLAYER_STATUSES,
stale: 'stale' as CalculatedPlayerStatus,
orphaned: 'orphaned' as CalculatedPlayerStatus,
}
export type ConfigureAsSource = 'source';
export type ConfigureAsClient = 'client';
export type ConfigureAs = ConfigureAsSource | ConfigureAsClient;
+1 -1
View File
@@ -1,4 +1,4 @@
import { type UnixTimestamp } from "../../../core/Atomic.ts"
import type {UnixTimestamp} from "../../../core/Atomic.ts";
export interface JobParameters {
/** maximum number of results to get for the entire job */
@@ -1,13 +1,13 @@
import { type LogOptions } from "@foxxmd/logging";
import { type ClientAIOConfig } from "./client/clients.ts";
import { type CommonClientOptions } from "./client/index.ts";
import { type RequestRetryOptions } from "./common.ts";
import { type WebhookConfig } from "./health/webhooks.ts";
import { type CommonSourceOptions, type SourceRetryOptions } from "./source/index.ts";
import { type SourceAIOConfig } from "./source/sources.ts";
import { type CacheConfigUser, type DurationValue } from "../Atomic.ts";
import { type TransformerCommonConfig } from "../../../../core/Atomic.ts";
import { type RetentionConfig } from "./database.ts";
import type { LogOptions } from "@foxxmd/logging";
import type { ClientAIOConfig } from "./client/clients.ts";
import type { CommonClientOptions } from "./client/index.ts";
import type { RequestRetryOptions } from "./common.ts";
import type { WebhookConfig } from "./health/webhooks.ts";
import type { CommonSourceOptions, SourceRetryOptions } from "./source/index.ts";
import type { SourceAIOConfig } from "./source/sources.ts";
import type { CacheConfigUser, DurationValue } from "../Atomic.ts";
import type { TransformerCommonConfig } from "../../../../core/Atomic.ts";
import type { RetentionConfig } from "./database.ts";
export interface SourceDefaults extends CommonSourceOptions {
@@ -1,4 +1,4 @@
import { type AtprotoDid } from "@atcute/lexicons/syntax";
import type {AtprotoDid} from "@atcute/lexicons/syntax";
export interface ATProtoUserIdentifierData {
/**
@@ -1,11 +1,11 @@
import { type KoitoClientAIOConfig, type KoitoClientConfig } from "./koito.ts";
import { type LastfmClientAIOConfig, type LastfmClientConfig } from "./lastfm.ts";
import { type ListenBrainzClientAIOConfig, type ListenBrainzClientConfig } from "./listenbrainz.ts";
import { type MalojaClientAIOConfig, type MalojaClientConfig } from "./maloja.ts";
import { type TealClientAIOConfig, type TealClientConfig } from "./tealfm.ts";
import { type RockSkyClientAIOConfig, type RockSkyClientConfig } from "./rocksky.ts";
import { type LibrefmClientConfig, type LibrefmClientAIOConfig } from "./librefm.ts";
import { type DiscordClientAIOConfig, type DiscordClientConfig } from "./discord.ts";
import type {KoitoClientAIOConfig, KoitoClientConfig} from "./koito.ts";
import type {LastfmClientAIOConfig, LastfmClientConfig} from "./lastfm.ts";
import type {ListenBrainzClientAIOConfig, ListenBrainzClientConfig} from "./listenbrainz.ts";
import type {MalojaClientAIOConfig, MalojaClientConfig} from "./maloja.ts";
import type {TealClientAIOConfig, TealClientConfig} from "./tealfm.ts";
import type {RockSkyClientAIOConfig, RockSkyClientConfig} from "./rocksky.ts";
import type {LibrefmClientConfig, LibrefmClientAIOConfig} from "./librefm.ts";
import type {DiscordClientAIOConfig, DiscordClientConfig} from "./discord.ts";
export type ClientConfig =
MalojaClientConfig
@@ -57,23 +57,3 @@ export const clientInterfaces = [
...atomicClientInterfaces
];
export type ClientType =
'maloja'
| 'lastfm'
| 'librefm'
| 'listenbrainz'
| 'koito'
| 'tealfm'
| 'rocksky'
| 'discord';
export const clientTypes: ClientType[] = [
'maloja',
'lastfm',
'librefm',
'listenbrainz',
'koito',
'tealfm',
'rocksky',
'discord'
];
@@ -1,5 +1,5 @@
import { type CommonClientConfig, type CommonClientData } from "./index.ts"
import { type ComponentType } from "../../../../../core/Atomic.ts"
import type {CommonClientConfig, CommonClientData} from "./index.ts";
import type {ComponentType} from "../../../../../core/Atomic.ts";
export interface DiscordData {
token?: string
@@ -1,7 +1,7 @@
import { type DurationValue } from "../../Atomic.ts";
import { type PlayTransformOptions } from "../../../../../core/Transform.ts";
import { type CommonConfig, type RequestRetryOptions } from "../common.ts";
import { type RetentionConfig } from "../database.ts";
import type {DurationValue} from "../../Atomic.ts";
import type {PlayTransformOptions} from "../../../../../core/Transform.ts";
import type {CommonConfig, RequestRetryOptions} from "../common.ts";
import type {RetentionConfig} from "../database.ts";
/**
* Scrobble matching (between new source track and existing client scrobbles) logging options. Used for debugging.
@@ -1,6 +1,6 @@
import { type ComponentType, type UnixTimestamp } from "../../../../../core/Atomic.ts"
import { type RequestRetryOptions } from "../common.ts"
import { type CommonClientConfig, type CommonClientData } from "./index.ts"
import type {ComponentType, UnixTimestamp} from "../../../../../core/Atomic.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData} from "./index.ts";
export interface ListensResponse {
items: ListenObjectResponse[]
@@ -1,5 +1,5 @@
import { type RequestRetryOptions } from "../common.ts";
import { type CommonClientConfig, type CommonClientData, type CommonClientOptions, type NowPlayingOptions } from "./index.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData, CommonClientOptions, NowPlayingOptions} from "./index.ts";
export interface LastfmData extends CommonClientData, RequestRetryOptions {
/**
@@ -1,6 +1,6 @@
import { type RequestRetryOptions } from "../common.ts";
import { type CommonClientConfig, type CommonClientData } from "./index.ts";
import { type LastfmClientOptions } from "./lastfm.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData} from "./index.ts";
import type {LastfmClientOptions} from "./lastfm.ts";
export interface LibrefmData extends CommonClientData, RequestRetryOptions {
/**
@@ -1,6 +1,6 @@
import { type ComponentType } from "../../../../../core/Atomic.ts";
import { type RequestRetryOptions } from "../common.ts";
import { type CommonClientConfig, type CommonClientData } from "./index.ts";
import type {ComponentType} from "../../../../../core/Atomic.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData} from "./index.ts";
export interface ListenBrainzData extends RequestRetryOptions{
/**
@@ -1,6 +1,6 @@
import { type ComponentType } from "../../../../../core/Atomic.ts";
import { type RequestRetryOptions } from "../common.ts";
import { type CommonClientConfig, type CommonClientData } from "./index.ts";
import type {ComponentType} from "../../../../../core/Atomic.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData} from "./index.ts";
export interface MalojaData extends RequestRetryOptions {
/**
@@ -1,5 +1,5 @@
import { type RequestRetryOptions } from "../common.ts";
import { type CommonClientConfig, type CommonClientData, type CommonClientOptions, type NowPlayingOptions } from "./index.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {CommonClientConfig, CommonClientData, CommonClientOptions, NowPlayingOptions} from "./index.ts";
export interface RockSkyData extends RequestRetryOptions{
@@ -1,7 +1,7 @@
import { type ComponentType } from "../../../../../core/Atomic.ts"
import { type RequestRetryOptions } from "../common.ts"
import { type ATProtoAppData, type ATProtoUserIdentifierData } from "./atproto.ts"
import { type CommonClientConfig, type CommonClientData, type CommonClientOptions } from "./index.ts"
import type {ComponentType} from "../../../../../core/Atomic.ts";
import type {RequestRetryOptions} from "../common.ts";
import type {ATProtoAppData, ATProtoUserIdentifierData} from "./atproto.ts";
import type {CommonClientConfig, CommonClientData, CommonClientOptions} from "./index.ts";
export type TealData = RequestRetryOptions & ATProtoUserIdentifierData & Partial<ATProtoAppData> & {
/**
@@ -1,5 +1,5 @@
import { type Duration } from "dayjs/plugin/duration.js";
import { type DurationValue } from "../Atomic.ts";
import type {Duration} from "dayjs/plugin/duration.js";
import type {DurationValue} from "../Atomic.ts";
export type RetentionPlayType = 'failed' | 'completed' | 'duped';
export const retentionPlayTypes: RetentionPlayType[] = ['failed','completed','duped'];
@@ -0,0 +1,58 @@
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface AppleMusicKey {
id: string
teamId: string
p8: string
}
export interface AppleMusicData extends CommonSourceData, PollingOptions {
key?: AppleMusicKey
token?: string
mediaUserToken?: string
/**
* Origin header to include in every Apple Music API request.
* Required when using a browser token (not a MusicKit key).
*
* @example "https://music.apple.com"
*/
origin?: string
}
export interface AppleMusicSourceConfig extends CommonSourceConfig {
data?: AppleMusicData
options?: CommonSourceOptions & {
logAuth?: boolean
logDiff?: boolean
/**
* Fixes a quirk where Apple Music's history API hides duplicate plays.
* If you listen to A → B → A, the API returns [A, B, X, Y ...] instead of [A, B, A, X, Y ...].
* This can cause MS to skip the second A play.
*
* When enabled (default), MS detects this pattern, keeps the interim tracks (B),
* and re-scrobbles A as a re-listen. Disable only if you notice false positives.
*
* @default true
* @examples [true, false]
*/
recoverUnchangedTopHistory?: boolean
/**
* Removes extraneous suffixes from album data
*
* Apple Music add ' - EP' and ' - Single' to album names for EP's and singles, respectively.
* These suffixes are not part of the official names for the album and can cause issues in scrobble services
* or when matching metadata (musicbrainz).
*
* When this option is true (default), Multi-scrobbler automatically removes these suffixes.
*
* @default true
* @examples [true, false]
*/
normalizeAlbum?: boolean
}
}
export interface AppleMusicSourceAIOConfig extends AppleMusicSourceConfig {
type: 'applemusic'
}
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions, type ManualListeningOptions } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions, ManualListeningOptions} from "./index.ts";
export interface AzuraStationInfoResponse {
id: string
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface ChromecastData extends CommonSourceData {
/**
@@ -1,6 +1,6 @@
import { type Second } from "../../../../../core/Atomic.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {Second} from "../../../../../core/Atomic.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface DeezerData extends CommonSourceData, PollingOptions {
/**
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface LastFMEndpointData extends CommonSourceData {
/**
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface ListenbrainzEndpointData extends CommonSourceData {
/**
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions, type ManualListeningOptions } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions, ManualListeningOptions} from "./index.ts";
export interface IcecastMetadata {
@@ -1,9 +1,9 @@
import { type FileLogOptions, type LogLevel } from "@foxxmd/logging";
import type { FileLogOptions, LogLevel } from "@foxxmd/logging";
import { type PlayTransformOptions } from "../../../../../core/Transform.ts";
import { type CommonConfig, type RequestRetryOptions } from "../common.ts";
import { type RetentionConfig } from "../database.ts";
import { type DurationValue } from "../../Atomic.ts";
import type { PlayTransformOptions } from "../../../../../core/Transform.ts";
import type { CommonConfig, RequestRetryOptions } from "../common.ts";
import type { RetentionConfig } from "../database.ts";
import type { DurationValue } from "../../Atomic.ts";
export interface SourceRetryOptions extends RequestRetryOptions {
/**
@@ -1,5 +1,5 @@
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import {
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
import type {
// @ts-expect-error weird typings?
CollectionType,
// @ts-expect-error weird typings?
@@ -1,5 +1,5 @@
import { type PollingOptions, type RequestRetryOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions, RequestRetryOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface JRiverData extends CommonSourceData, PollingOptions, RequestRetryOptions {
/**
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface KodiData extends CommonSourceData, PollingOptions {
@@ -1,6 +1,6 @@
import { type KoitoData } from "../client/koito.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {KoitoData} from "../client/koito.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface KoitoSourceData extends KoitoData, CommonSourceData, PollingOptions {
}
@@ -1,6 +1,6 @@
import { type LastfmData } from "../client/lastfm.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {LastfmData} from "../client/lastfm.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface LastFmSourceData extends CommonSourceData, PollingOptions, LastfmData{}
@@ -1,6 +1,6 @@
import { type LibrefmData } from "../client/librefm.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {LibrefmData} from "../client/librefm.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface librefmSourceData extends CommonSourceData, PollingOptions, LibrefmData{}
@@ -1,6 +1,6 @@
import { type ListenBrainzData } from "../client/listenbrainz.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {ListenBrainzData} from "../client/listenbrainz.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface ListenBrainzSourceData extends ListenBrainzData, CommonSourceData, PollingOptions {
}
@@ -1,6 +1,6 @@
import { type MalojaData } from "../client/maloja.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {MalojaData} from "../client/maloja.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface MalojaSourceData extends MalojaData, CommonSourceData, PollingOptions {
}
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface MopidyData extends CommonSourceData, PollingOptions {
/**
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface MPDData extends CommonSourceData, PollingOptions {
/**
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export const PLAYBACK_STATUS_PLAYING = 'Playing';
export const PLAYBACK_STATUS_PAUSED = 'Paused';
@@ -1,5 +1,6 @@
import { REPORTED_PLAYER_STATUSES, type ReportedPlayerStatus } from "../../Atomic.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import { REPORTED_PLAYER_STATUSES } from '../../../../../core/Atomic.ts';
import type {ReportedPlayerStatus} from '../../../../../core/Atomic.ts';
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export type PlaybackStatus = 'play' | 'stop' | 'pause' | 'fast_reverse' | 'fast_forward'
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export const PLAYBACK_STATUS_PLAYING_MC = 'playing';
export const PLAYBACK_STATUS_PAUSED_MC = 'paused';
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface PlexApiData extends CommonSourceData, PollingOptions {
token?: string
@@ -1,6 +1,6 @@
import { type RockSkyData, type RockSkyOptions } from "../client/rocksky.ts";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {RockSkyData, RockSkyOptions} from "../client/rocksky.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface RockskySourceData extends RockSkyData, CommonSourceData, PollingOptions {
}
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface SonosData extends CommonSourceData, PollingOptions {
/**
@@ -1,32 +1,33 @@
import { type AzuracastSourceAIOConfig, type AzuracastSourceConfig } from "./azuracast.ts";
import { type ChromecastSourceAIOConfig, type ChromecastSourceConfig } from "./chromecast.ts";
import { type ListenbrainzEndpointSourceAIOConfig, type ListenbrainzEndpointSourceConfig } from "./endpointlz.ts";
import { type LastFMEndpointSourceAIOConfig, type LastFMEndpointSourceConfig } from "./endpointlfm.ts";
import { type DeezerInternalSourceConfig, type DeezerSourceConfig, type DeezerCompatConfig, type DeezerAIOCompatConfig } from "./deezer.ts";
import { type JellyApiSourceAIOConfig, type JellyApiSourceConfig } from "./jellyfin.ts";
import { type JRiverSourceAIOConfig, type JRiverSourceConfig } from "./jriver.ts";
import { type KodiSourceAIOConfig, type KodiSourceConfig } from "./kodi.ts";
import { type LastFmSouceAIOConfig, type LastfmSourceConfig } from "./lastfm.ts";
import { type ListenBrainzSourceAIOConfig, type ListenBrainzSourceConfig } from "./listenbrainz.ts";
import { type MopidySourceAIOConfig, type MopidySourceConfig } from "./mopidy.ts";
import { type MPDSourceAIOConfig, type MPDSourceConfig } from "./mpd.ts";
import { type MPRISSourceAIOConfig, type MPRISSourceConfig } from "./mpris.ts";
import { type MusikcubeSourceAIOConfig, type MusikcubeSourceConfig } from "./musikcube.ts";
import { type MusicCastSourceConfig, type MusicCastSourceAIOConfig } from "./musiccast.ts";
import { type PlexApiSourceConfig, type PlexApiSourceAIOConfig } from "./plex.ts";
import { type SpotifySourceAIOConfig, type SpotifySourceConfig } from "./spotify.ts";
import { type SubsonicSourceAIOConfig, type SubSonicSourceConfig } from "./subsonic.ts";
import { type VLCSourceAIOConfig, type VLCSourceConfig } from "./vlc.ts";
import { type WebScrobblerSourceAIOConfig, type WebScrobblerSourceConfig } from "./webscrobbler.ts";
import { type YTMusicSourceAIOConfig, type YTMusicSourceConfig } from "./ytmusic.ts";
import { type YandexMusicBridgeSourceAIOConfig, type YandexMusicBridgeSourceConfig } from "./ymbridge.ts";
import { type IcecastSourceAIOConfig, type IcecastSourceConfig } from "./icecast.ts";
import { type KoitoSourceAIOConfig, type KoitoSourceConfig } from "./koito.ts";
import { type MalojaSourceAIOConfig, type MalojaSourceConfig } from "./maloja.ts";
import { type TealSourceAIOConfig, type TealSourceConfig } from "./tealfm.ts";
import { type RockskySourceAIOConfig, type RockskySourceConfig } from "./rocksky.ts";
import { type LibrefmSouceAIOConfig, type LibrefmSourceConfig } from "./librefm.ts";
import { type SonosSourceAIOConfig, type SonosSourceConfig } from "./sonos.ts";
import type {AzuracastSourceAIOConfig, AzuracastSourceConfig} from "./azuracast.ts";
import type {ChromecastSourceAIOConfig, ChromecastSourceConfig} from "./chromecast.ts";
import type {ListenbrainzEndpointSourceAIOConfig, ListenbrainzEndpointSourceConfig} from "./endpointlz.ts";
import type {LastFMEndpointSourceAIOConfig, LastFMEndpointSourceConfig} from "./endpointlfm.ts";
import type {DeezerInternalSourceConfig, DeezerSourceConfig, DeezerCompatConfig, DeezerAIOCompatConfig} from "./deezer.ts";
import type {JellyApiSourceAIOConfig, JellyApiSourceConfig} from "./jellyfin.ts";
import type {JRiverSourceAIOConfig, JRiverSourceConfig} from "./jriver.ts";
import type {KodiSourceAIOConfig, KodiSourceConfig} from "./kodi.ts";
import type {LastFmSouceAIOConfig, LastfmSourceConfig} from "./lastfm.ts";
import type {ListenBrainzSourceAIOConfig, ListenBrainzSourceConfig} from "./listenbrainz.ts";
import type {MopidySourceAIOConfig, MopidySourceConfig} from "./mopidy.ts";
import type {MPDSourceAIOConfig, MPDSourceConfig} from "./mpd.ts";
import type {MPRISSourceAIOConfig, MPRISSourceConfig} from "./mpris.ts";
import type {MusikcubeSourceAIOConfig, MusikcubeSourceConfig} from "./musikcube.ts";
import type {MusicCastSourceConfig, MusicCastSourceAIOConfig} from "./musiccast.ts";
import type {PlexApiSourceConfig, PlexApiSourceAIOConfig} from "./plex.ts";
import type {SpotifySourceAIOConfig, SpotifySourceConfig} from "./spotify.ts";
import type {SubsonicSourceAIOConfig, SubSonicSourceConfig} from "./subsonic.ts";
import type {VLCSourceAIOConfig, VLCSourceConfig} from "./vlc.ts";
import type {WebScrobblerSourceAIOConfig, WebScrobblerSourceConfig} from "./webscrobbler.ts";
import type {YTMusicSourceAIOConfig, YTMusicSourceConfig} from "./ytmusic.ts";
import type {YandexMusicBridgeSourceAIOConfig, YandexMusicBridgeSourceConfig} from "./ymbridge.ts";
import type {IcecastSourceAIOConfig, IcecastSourceConfig} from "./icecast.ts";
import type {KoitoSourceAIOConfig, KoitoSourceConfig} from "./koito.ts";
import type {MalojaSourceAIOConfig, MalojaSourceConfig} from "./maloja.ts";
import type {TealSourceAIOConfig, TealSourceConfig} from "./tealfm.ts";
import type {RockskySourceAIOConfig, RockskySourceConfig} from "./rocksky.ts";
import type {LibrefmSouceAIOConfig, LibrefmSourceConfig} from "./librefm.ts";
import type {SonosSourceAIOConfig, SonosSourceConfig} from "./sonos.ts";
import type {AppleMusicSourceAIOConfig, AppleMusicSourceConfig} from "./applemusic.ts";
export type SourceConfig =
@@ -58,7 +59,8 @@ export type SourceConfig =
| KoitoSourceConfig
| TealSourceConfig
| RockskySourceConfig
| SonosSourceConfig;
| SonosSourceConfig
| AppleMusicSourceConfig;
export type SourceAIOConfig =
SpotifySourceAIOConfig
@@ -89,7 +91,8 @@ export type SourceAIOConfig =
| KoitoSourceAIOConfig
| TealSourceAIOConfig
| RockskySourceAIOConfig
| SonosSourceAIOConfig;
| SonosSourceAIOConfig
| AppleMusicSourceAIOConfig;
/** Used for docusaurus schemas
* We need to show "array of" for each type of config when looking at File Config
@@ -128,71 +131,9 @@ export type KoitoSourceConfigs = KoitoSourceConfig[];
export type TealSourceConfigs = TealSourceConfig[];
export type RockskySourceConfigs = RockskySourceConfig[];
export type SonosSourceConfigs = SonosSourceConfig[];
export type AppleMusicSourceConfigs = AppleMusicSourceConfig[];
export type SourceType =
'spotify'
| 'plex'
| 'subsonic'
| 'jellyfin'
| 'lastfm'
| 'librefm'
| 'deezer'
| 'endpointlz'
| 'endpointlfm'
| 'ytmusic'
| 'ymbridge'
| 'mpris'
| 'mopidy'
| 'musiccast'
| 'listenbrainz'
| 'jriver'
| 'kodi'
| 'webscrobbler'
| 'chromecast'
| 'maloja'
| 'musikcube'
| 'mpd'
| 'vlc'
| 'icecast'
| 'azuracast'
| 'koito'
| 'tealfm'
| 'rocksky'
| 'sonos';
export const sourceTypes: SourceType[] = [
'spotify',
'plex',
'subsonic',
'jellyfin',
'lastfm',
'librefm',
'deezer',
'endpointlz',
'endpointlfm',
'ytmusic',
'ymbridge',
'mpris',
'mopidy',
'musiccast',
'listenbrainz',
'jriver',
'kodi',
'webscrobbler',
'chromecast',
'maloja',
'musikcube',
'mpd',
'vlc',
'icecast',
'azuracast',
'koito',
'tealfm',
'rocksky',
'sonos'
];
export const atomicSourceInterfaces = [
'SpotifySourceConfig',
'PlexApiSourceConfig',
@@ -222,7 +163,8 @@ export const atomicSourceInterfaces = [
'KoitoSourceConfig',
'TealSourceConfig',
'RockskySourceConfig',
'SonosSourceConfig'
'SonosSourceConfig',
'AppleMusicSourceConfig'
];
export const sourceInterfaces = [
@@ -230,7 +172,4 @@ export const sourceInterfaces = [
...atomicSourceInterfaces
];
export const isSourceType = (data: string): data is SourceType => {
return sourceTypes.includes(data as SourceType);
};
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface SpotifySourceData extends CommonSourceData, PollingOptions {
/**
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface SubsonicData extends CommonSourceData, PollingOptions {
/**
@@ -1,6 +1,6 @@
import { type TealData, type TealOptions } from "../client/tealfm.ts"
import { type PollingOptions } from "../common.ts"
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts"
import type {TealData, TealOptions} from "../client/tealfm.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface TealSourceData extends TealData, CommonSourceData, PollingOptions {
@@ -1,6 +1,6 @@
import { type VlcMeta } from "vlc-client/dist/Types.js";
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {VlcMeta} from "vlc-client/dist/Types.js";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface VLCData extends CommonSourceData, PollingOptions {
/**
@@ -1,4 +1,4 @@
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface WebScrobblerData extends CommonSourceData {
/**
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData} from "./index.ts";
export interface YandexMusicBridgeData extends CommonSourceData, PollingOptions {
/** URL of the local Python bridge, for example http://yandex-music-bridge:9980 */
@@ -1,5 +1,5 @@
import { type PollingOptions } from "../common.ts";
import { type CommonSourceConfig, type CommonSourceData, type CommonSourceOptions } from "./index.ts";
import type {PollingOptions} from "../common.ts";
import type {CommonSourceConfig, CommonSourceData, CommonSourceOptions} from "./index.ts";
export interface InnertubeOptions {
/**
+8 -10
View File
@@ -1,15 +1,13 @@
import { type FileLogOptions, type Logger, loggerAppRolling, type LogLevel, type LogLevelStreamEntry, type LogOptions, parseLogOptions } from '@foxxmd/logging';
import { buildDestinationJsonPrettyStream, buildDestinationRollingFile, buildDestinationStdout, buildLogger } from "@foxxmd/logging/factory";
import { PassThrough, Transform } from "node:stream";
import type { Transform } from "node:stream";
import { PassThrough } from "node:stream";
import path from "path";
import process from "process";
import { projectDir } from "./index.ts";
import { getDataDir } from "./index.ts";
import { isDebugMode } from '../utils.ts';
export let logPath = path.resolve(projectDir, `./logs`);
if (typeof process.env.CONFIG_DIR === 'string') {
logPath = path.resolve(process.env.CONFIG_DIR, './logs');
}
const logPath = path.resolve(getDataDir(), `./logs`);
export const initLogger = (): [Logger, Transform] => {
const opts = parseLogOptions({file: false, console: 'trace'})
@@ -26,8 +24,8 @@ export const appLogger = async (config: LogOptions = {}): Promise<[Logger, PassT
const { file } = config;
const opts = parseLogOptions(isDebugMode() ? {...config, file: typeof file === 'object' ? {...file, level: 'trace'} : 'trace', console: 'trace', level: 'trace'} : config);
const logger = await loggerAppRolling(opts, {
logBaseDir: typeof process.env.CONFIG_DIR === 'string' ? process.env.CONFIG_DIR : undefined,
logDefaultPath: './logs/scrobble.log',
logBaseDir: logPath,
logDefaultPath: './scrobble.log',
destinations: [
buildDestinationJsonPrettyStream('trace', {destination: stream, object: true, colorize: true})
]
@@ -37,8 +35,8 @@ export const appLogger = async (config: LogOptions = {}): Promise<[Logger, PassT
export const componentFileLogger = async (type: string, name: string, fileConfig: true | LogLevel | FileLogOptions, config: LogOptions = {}): Promise<Logger> => {
const opts = parseLogOptions(config, {
logBaseDir: typeof process.env.CONFIG_DIR === 'string' ? process.env.CONFIG_DIR : undefined,
logDefaultPath: './logs/scrobble.log'
logBaseDir: logPath,
logDefaultPath: './scrobble.log'
});
const base = path.dirname(typeof opts.file.path === 'function' ? opts.file.path() : opts.file.path);
@@ -1,15 +1,16 @@
import { childLogger, type Logger } from "@foxxmd/logging";
import { type PlayObject, type TransformerCommon, type TransformerCommonConfig } from "../../../core/Atomic.ts";
import type { PlayObject, TransformerCommon, TransformerCommonConfig } from "../../../core/Atomic.ts";
import { isStageTyped, testWhenConditions } from "../../utils/PlayTransformUtils.ts";
import AbstractInitializable from "../AbstractInitializable.ts";
import { type StageConfig } from "../../../core/Transform.ts";
import { cacheFunctions, parseToRegexOrLiteralSearch, testMaybeRegex, searchAndReplace} from "@foxxmd/regex-buddy-core";
import { Cacheable } from "cacheable";
import type { StageConfig } from "../../../core/Transform.ts";
import type { cacheFunctions} from "@foxxmd/regex-buddy-core";
import { parseToRegexOrLiteralSearch, testMaybeRegex, searchAndReplace} from "@foxxmd/regex-buddy-core";
import type { Cacheable } from "cacheable";
import { hashObject } from "../../utils/StringUtils.ts";
import { playContentInvariantTransform } from "../../utils/PlayComparisonUtils.ts";
import { SkipTransformStageError, StagePrerequisiteError } from "../errors/MSErrors.ts";
import { capitalize } from "../../../core/StringUtils.ts";
import { type StaggerOptions } from "../../utils/AsyncUtils.ts";
import type { StaggerOptions } from "../../utils/AsyncUtils.ts";
export interface TransformerOptions {
logger: Logger
@@ -1,5 +1,5 @@
import { type ArtistCredit, isPlayObject, type ObjectPlayData, type PlayObject, type TrackMeta } from "../../../core/Atomic.ts";
import { type AtomicStageConfig, type StageConfig } from "../../../core/Transform.ts";
import type {AtomicStageConfig, StageConfig} from "../../../core/Transform.ts";
import AbstractTransformer from "./AbstractTransformer.ts";
//export type GenericAtomicStageConfig<A> =
@@ -1,21 +1,21 @@
import { type ArtistCredit, asMBReleasePrimaryGroupType, asMBReleaseSecondaryGroupType, asMBReleaseStatus, DEFAULT_MISSING_TYPES, type LifecycleInput, type MBReleaseGroupPrimaryType, type MBReleaseGroupSecondaryType, type MBReleaseStatus, type MissingMbidType, type PlayObject, type TrackMeta, type TransformerCommon, type TransformOptions } from "../../../core/Atomic.ts";
import { isWhenCondition, testWhenConditions } from "../../utils/PlayTransformUtils.ts";
import { type WebhookPayload } from "../infrastructure/config/health/webhooks.ts";
import { type ExternalMetadataTerm, type PlayTransformMetadataStage } from "../../../core/Transform.ts";
import type {WebhookPayload} from "../infrastructure/config/health/webhooks.ts";
import type {ExternalMetadataTerm, PlayTransformMetadataStage} from "../../../core/Transform.ts";
import AtomicPartsTransformer from "./AtomicPartsTransformer.ts";
import { type TransformerOptions } from "./AbstractTransformer.ts";
import type {TransformerOptions} from "./AbstractTransformer.ts";
import { ARTIST_WEIGHT, type MusicbrainzApiConfigData, TITLE_WEIGHT } from "../infrastructure/Atomic.ts";
import { DELIMITERS } from '../../../core/Atomic.ts';
import { MaybeLogger } from '../MaybeLogger.ts';
import { childLogger } from "@foxxmd/logging";
import { MusicbrainzApiClient, recordingToPlay, type UsingTypes } from "../vendor/musicbrainz/MusicbrainzApiClient.ts";
import { type IRecordingList, type IRecordingMatch } from "musicbrainz-api";
import type {IRecordingList, IRecordingMatch} from "musicbrainz-api";
import { intersect, missingMbidTypes } from "../../utils.ts";
import { removeUndefinedKeys } from '../../../core/DataUtils.ts';
import { SimpleError, SkipTransformStageError, StagePrerequisiteError, StageTransformError } from "../errors/MSErrors.ts";
import { parseArrayFromMaybeString, scoreNormalizedStringsWeighted } from "../../utils/StringUtils.ts";
import clone from "clone";
import { Cacheable } from "cacheable";
import type { Cacheable } from "cacheable";
import { splitByFirstRegexFound } from "../../../core/StringUtils.ts";
import { nativeParse } from "./NativeTransformer.ts";
import { comparePlayArtistsNormalized, scoreTrackWeightedAndNormalized } from "../../utils/PlayComparisonUtils.ts";
@@ -435,7 +435,7 @@ export default class MusicbrainzTransformer extends AtomicPartsTransformer<Exter
// preserve order of search from before searchOrder
searchOrder = this.defaults.searchOrder ?? ['isrc', 'basic']
} = stageConfig;
let results: IRecordingMSList;
const queries: LifecycleInput[] = [];
@@ -472,6 +472,7 @@ export default class MusicbrainzTransformer extends AtomicPartsTransformer<Exter
}
} catch (e) {
if(e instanceof SearchPrerequisiteError) {
queries.push({type: `mbQuery-${searchType}-prereqFailure`, input: `Search type ${searchType} did not meet prerequesites: ${e.message}`});
this.logger.debug(`Search type ${searchType} did not meet prerequesites: ${e.message}`);
} else {
// we should be catching any unrecoverable errors in api calls
@@ -601,6 +602,14 @@ export default class MusicbrainzTransformer extends AtomicPartsTransformer<Exter
}
public async handlePostFetch(play: PlayObject, transformData: IRecordingMSList, stageConfig: MusicbrainzTransformerDataStage): Promise<PlayObject> {
// if all searches fail prereqs then no recording lists are assigned to results
if(transformData.recordings === undefined) {
throw new StagePrerequisiteError('All search prerequisites failed, Musicbrainz API could not be searched with the given searchOrder options',
{
shortStack: true,
inputs: transformData.requestQueries
});
}
if(transformData.recordings.length === 0) {
throw new StagePrerequisiteError('No matches returned from Musicbrainz API', {shortStack: true, inputs: transformData.requestQueries});
}
@@ -611,7 +620,7 @@ export default class MusicbrainzTransformer extends AtomicPartsTransformer<Exter
// if brainz meta contains track MBID then we should be able to get the exact release
let explicitList: IRecordingMatch[];
let filtered = false;
let filtered: boolean;
[explicitList, filtered] = filterByExplicitTrackMbid(transformData.recordings, play);
if(filtered) {
this.logger.debug(`Found exact release using track MBID`);
@@ -752,8 +761,7 @@ export const filterByValidReleaseStatus = <T extends IRecordingMatch[]>(list: T,
if(releaseStatusAllow.length === 0 && releaseStatusDeny.length === 0) {
return list;
}
const releaseFiltered = list.map(x => {
return {
const releaseFiltered = list.map(x => ({
...x,
releases: x.releases === undefined ? [] : x.releases.filter(y => {
if(releaseStatusAllow.length > 0) {
@@ -761,8 +769,7 @@ export const filterByValidReleaseStatus = <T extends IRecordingMatch[]>(list: T,
}
return !releaseStatusDeny.includes(y.status?.toLocaleLowerCase() as MBReleaseStatus)
})
}
});
}));
return releaseFiltered.filter(x => (
(list.find(y => y.id === x.id).releases ?? []).length === 0
&& releaseAllowEmpty
@@ -779,8 +786,7 @@ export const filterByValidReleaseGroupPrimary = <T extends IRecordingMatch[]>(li
if(releaseGroupPrimaryTypeAllow.length === 0 && releaseGroupPrimaryTypeAllow.length === 0) {
return list;
}
const releaseFiltered = list.map(x => {
return {
const releaseFiltered = list.map(x => ({
...x,
releases: x.releases === undefined ? [] : x.releases.filter(y => {
if(releaseGroupPrimaryTypeAllow.length > 0) {
@@ -788,8 +794,7 @@ export const filterByValidReleaseGroupPrimary = <T extends IRecordingMatch[]>(li
}
return !releaseGroupPrimaryTypeDeny.includes(y["release-group"]?.["primary-type"]?.toLocaleLowerCase() as MBReleaseGroupPrimaryType)
})
}
});
}));
return releaseFiltered.filter(x => (
(list.find(y => y.id === x.id).releases ?? []).length === 0
&& releaseAllowEmpty
@@ -806,8 +811,7 @@ export const filterByValidReleaseGroupSecondary = (list: IRecordingMatch[], stag
if(releaseGroupSecondaryTypeAllow.length === 0 && releaseGroupSecondaryTypeDeny.length === 0) {
return list;
}
const releaseFiltered = list.map(x => {
return {
const releaseFiltered = list.map(x => ({
...x,
releases: x.releases === undefined ? [] : x.releases.filter(y => {
if(releaseGroupSecondaryTypeAllow.length > 0) {
@@ -815,8 +819,7 @@ export const filterByValidReleaseGroupSecondary = (list: IRecordingMatch[], stag
}
return intersect(releaseGroupSecondaryTypeDeny, (y["release-group"]?.["secondary-types"] ?? []).map(x => x.toLocaleLowerCase()) as MBReleaseGroupSecondaryType[]).length === 0;
})
}
});
}));
return releaseFiltered.filter(x => (
(list.find(y => y.id === x.id).releases ?? []).length === 0
&& releaseAllowEmpty
@@ -833,8 +836,7 @@ export const filterByValidReleaseCountry = (list: IRecordingMatch[], stageConfig
if(releaseCountryAllow.length === 0 && releaseCountryDeny.length === 0) {
return list;
}
const releaseFiltered = list.map(x => {
return {
const releaseFiltered = list.map(x => ({
...x,
releases: x.releases === undefined ? [] : x.releases.filter(y => {
if(releaseCountryAllow.length > 0) {
@@ -842,8 +844,7 @@ export const filterByValidReleaseCountry = (list: IRecordingMatch[], stageConfig
}
return !releaseCountryDeny.includes(y.country?.toLocaleLowerCase())
})
}
});
}));
return releaseFiltered.filter(x => (
(list.find(y => y.id === x.id).releases ?? []).length === 0
&& releaseAllowEmpty
@@ -862,8 +863,7 @@ export const filterByExplicitTrackMbid = (list: IRecordingMatch[], play: PlayObj
break;
}
for (const rel of rec.releases) {
// @ts-ignore
if (rel.media.some(x => x.track.some(y => y.id === play.data.meta.brainz.track))) {
if (rel.media.some(x => x.tracks.some(y => y.id === play.data.meta.brainz.track))) {
releaseMatchId = rel.id;
recMatch = rec;
break;
@@ -1,11 +1,11 @@
import { type ArtistCredit, type PlayObject, type TransformerCommon } from "../../../core/Atomic.ts";
import type {ArtistCredit, PlayObject, TransformerCommon} from "../../../core/Atomic.ts";
import { isWhenCondition, testWhenConditions } from "../../utils/PlayTransformUtils.ts";
import { type WebhookPayload } from "../infrastructure/config/health/webhooks.ts";
import { type ExternalMetadataTerm, type PlayTransformNativeStage } from "../../../core/Transform.ts";
import type {WebhookPayload} from "../infrastructure/config/health/webhooks.ts";
import type {ExternalMetadataTerm, PlayTransformNativeStage} from "../../../core/Transform.ts";
import AtomicPartsTransformer from "./AtomicPartsTransformer.ts";
import { parseArtistCredits, parseTrackCredits, uniqueNormalizedStrArr } from "../../utils/StringUtils.ts";
import { parseRegexSingle, parseToRegexOrLiteralSearch } from "@foxxmd/regex-buddy-core";
import { type TransformerOptions } from "./AbstractTransformer.ts";
import type {TransformerOptions} from "./AbstractTransformer.ts";
import { DELIMITERS_NO_AMP } from '../../../core/Atomic.ts';
import { asArray } from "../../utils/DataUtils.ts";
import { MaybeLogger } from '../MaybeLogger.ts';
@@ -1,11 +1,11 @@
import { childLogger, type Logger } from "@foxxmd/logging";
import AbstractTransformer from "./AbstractTransformer.ts";
import { type TransformerCommonConfig } from "../../../core/Atomic.ts";
import type AbstractTransformer from "./AbstractTransformer.ts";
import type {TransformerCommonConfig} from "../../../core/Atomic.ts";
import UserTransformer from "./UserTransformer.ts";
import { type StageConfig } from "../../../core/Transform.ts";
import { type PlayObject } from "../../../core/Atomic.ts";
import type {StageConfig} from "../../../core/Transform.ts";
import type {PlayObject} from "../../../core/Atomic.ts";
import { isStageTyped } from "../../utils/PlayTransformUtils.ts";
import { MSCache } from "../Cache.ts";
import type { MSCache } from "../Cache.ts";
import NativeTransformer from "./NativeTransformer.ts";
import MusicbrainzTransformer, { configFromEnv, type MusicbrainzTransformerConfig } from "./MusicbrainzTransformer.ts";
import { AsyncLocalStorage } from 'node:async_hooks';
@@ -1,8 +1,8 @@
import { searchAndReplace } from "@foxxmd/regex-buddy-core";
import { type ArtistCredit, type PlayObject } from "../../../core/Atomic.ts";
import type {ArtistCredit, PlayObject} from "../../../core/Atomic.ts";
import { configValToSearchReplace, isSearchAndReplaceTerm, isUserStage, testWhenConditions } from "../../utils/PlayTransformUtils.ts";
import { type WebhookPayload } from "../infrastructure/config/health/webhooks.ts";
import { type ConditionalSearchAndReplaceRegExp, type PlayTransformUserStage } from "../../../core/Transform.ts";
import type {WebhookPayload} from "../infrastructure/config/health/webhooks.ts";
import type {ConditionalSearchAndReplaceRegExp, PlayTransformUserStage} from "../../../core/Transform.ts";
import AtomicPartsTransformer from "./AtomicPartsTransformer.ts";
export default class UserTransformer extends AtomicPartsTransformer<ConditionalSearchAndReplaceRegExp[], undefined> {
+2 -2
View File
@@ -1,7 +1,7 @@
import { childLogger, type Logger } from "@foxxmd/logging";
import { type PlayObject } from "../../../core/Atomic.ts";
import type { PlayObject } from "../../../core/Atomic.ts";
import { capitalize } from "../../../core/StringUtils.ts";
import { type AbstractApiOptions, type FormatPlayObjectOptions } from "../infrastructure/Atomic.ts";
import type { AbstractApiOptions, FormatPlayObjectOptions } from "../infrastructure/Atomic.ts";
export default abstract class AbstractApiClient {
name: string;
+3 -2
View File
@@ -1,7 +1,8 @@
import request, { Request, Response } from 'superagent';
import type { Request, Response } from 'superagent';
import request from 'superagent';
import xml2js from 'xml2js';
import { type AbstractApiOptions, DEFAULT_RETRY_MULTIPLIER } from "../infrastructure/Atomic.ts";
import { type JRiverData } from "../infrastructure/config/source/jriver.ts";
import type {JRiverData} from "../infrastructure/config/source/jriver.ts";
import AbstractApiClient from "./AbstractApiClient.ts";
const parser = new xml2js.Parser({'async': true});
+4 -4
View File
@@ -2,10 +2,10 @@ import dayjs from "dayjs";
import { KodiClient } from 'kodi-api'
import normalizeUrl from "normalize-url";
import { URL } from "url";
import { type PlayObject, type PlayObjectMinimal } from "../../../core/Atomic.ts";
import { type RecentlyPlayedOptions } from "../../sources/AbstractSource.ts";
import { type AbstractApiOptions, type FormatPlayObjectOptions } from "../infrastructure/Atomic.ts";
import { type KodiData } from "../infrastructure/config/source/kodi.ts";
import type {PlayObject, PlayObjectMinimal} from "../../../core/Atomic.ts";
import type {RecentlyPlayedOptions} from "../../sources/AbstractSource.ts";
import type {AbstractApiOptions, FormatPlayObjectOptions} from "../infrastructure/Atomic.ts";
import type {KodiData} from "../infrastructure/config/source/kodi.ts";
import AbstractApiClient from "./AbstractApiClient.ts";
import { baseFormatPlayObj } from "../../utils/PlayTransformUtils.ts";
import { artistNamesToCredits } from "../../../core/StringUtils.ts";
+4 -4
View File
@@ -1,5 +1,5 @@
import dayjs, { type Dayjs, type ManipulateType } from "dayjs";
import { type BrainzMeta, type PlayObject, type PlayObjectMinimal, type ScrobbleActionResult, type UnixTimestamp, type URLData, type Writeable } from "../../../core/Atomic.ts";
import type {BrainzMeta, PlayObject, PlayObjectMinimal, ScrobbleActionResult, UnixTimestamp, URLData, Writeable} from "../../../core/Atomic.ts";
import { artistNamesToCredits, artistNameToCredit, nonEmptyStringOrDefault, splitByFirstFound } from "../../../core/StringUtils.ts";
import { sleep } from "../../utils.ts";
import { removeUndefinedKeys } from '../../../core/DataUtils.ts';
@@ -10,12 +10,12 @@ import { getScrobbleTsSOCDate } from "../../utils/TimeUtils.ts";
import { getNodeNetworkException, isNodeNetworkException } from "../errors/NodeErrors.ts";
import { UpstreamError } from "../errors/UpstreamError.ts";
import { type AbstractApiOptions, DEFAULT_RETRY_MULTIPLIER, type FormatPlayObjectOptions, type InternalConfigOptional, type PaginatedListensTimeRangeOptions, type PaginatedTimeRangeListens, type PaginatedTimeRangeListensResult } from "../infrastructure/Atomic.ts";
import { type LastfmData } from "../infrastructure/config/client/lastfm.ts";
import type {LastfmData} from "../infrastructure/config/client/lastfm.ts";
import AbstractApiClient from "./AbstractApiClient.ts";
import { normalizeStr, parseArtistCredits } from "../../utils/StringUtils.ts";
import { LastFMUser, LastFMAuth, LastFMTrack, type LastFMUserGetRecentTracksResponse, type LastFMBooleanNumber, type LastFMUpdateNowPlayingResponse, type LastFMUserGetInfoResponse, type LastFMUserGetRecentTracksParams } from 'lastfm-ts-api';
import clone from 'clone';
import { IncomingMessage } from "http";
import type { IncomingMessage } from "http";
import { baseFormatPlayObj } from "../../utils/PlayTransformUtils.ts";
import { ScrobbleSubmitError, SimpleError } from "../errors/MSErrors.ts";
import { redactString } from "@foxxmd/redact-string";
@@ -237,7 +237,7 @@ export default class LastfmApiClient extends AbstractApiClient implements Pagina
return true;
} catch (e) {
const hint = e.error?.cause?.message ?? undefined;
// eslint-disable-next-line preserve-caught-error
throw new Error(`Could not connect to ${this.upstreamName} API server${hint !== undefined ? ` (${hint})` : ''}`, { cause: e.error ?? e });
}
}
+5 -4
View File
@@ -1,7 +1,8 @@
import { stringSameness } from '@foxxmd/string-sameness';
import dayjs from "dayjs";
import request, { Request, Response } from 'superagent';
import { type BrainzMeta, type PlayObject, type PlayObjectMinimal, type ScrobbleActionResult, type UnixTimestamp, type URLData } from "../../../core/Atomic.ts";
import type { Request, Response } from 'superagent';
import request from 'superagent';
import type {BrainzMeta, PlayObject, PlayObjectMinimal, ScrobbleActionResult, UnixTimestamp, URLData} from "../../../core/Atomic.ts";
import { artistNamesToCredits, combinePartsToString, slice } from "../../../core/StringUtils.ts";
import {
normalizeListenbrainzUrl,
@@ -20,7 +21,7 @@ import AbstractApiClient from "./AbstractApiClient.ts";
import { getBaseFromUrl, isPortReachableConnect, joinedUrl, normalizeWebAddress } from '../../utils/NetworkUtils.ts';
import { unique } from '../../utils.ts';
import { removeUndefinedKeys } from '../../../core/DataUtils.ts';
import { type ListenPayload, type ListenResponse, type ListenType, type SubmitPayload } from '../../../core/vendor/listenbrainz/interfaces.ts';
import type {ListenPayload, ListenResponse, ListenType, SubmitPayload} from '../../../core/vendor/listenbrainz/interfaces.ts';
import { baseFormatPlayObj } from '../../utils/PlayTransformUtils.ts';
import { ScrobbleSubmitError, SimpleError } from '../errors/MSErrors.ts';
import pRetry from 'p-retry';
@@ -288,7 +289,7 @@ export class ListenbrainzApiClient extends AbstractApiClient implements Pageless
try {
const lr = await this.getUserListens({...lzListensOptions, user});
const more = to > lr.latest_listen_ts;
const more = lr.count > 0 && lr.count === lzListensOptions.count;
return {data: lr.listens.map(x => listenResponseToPlay(x)), meta: {...options, total: lr.count, more}};
} catch (e) {
throw e;
+10 -9
View File
@@ -1,24 +1,25 @@
import dayjs from "dayjs";
import request, { Request, Response } from 'superagent';
import { type PlayObject, type PlayObjectMinimal, type ScrobbleActionResult, type URLData } from "../../../core/Atomic.ts";
import type { Request, Response } from 'superagent';
import request from 'superagent';
import type {PlayObject, PlayObjectMinimal, ScrobbleActionResult, URLData} from "../../../core/Atomic.ts";
import { artistCreditsToNames, artistNamesToCredits, nonEmptyStringOrDefault } from "../../../core/StringUtils.ts";
import { UpstreamError } from "../errors/UpstreamError.ts";
import { type AbstractApiOptions, type FormatPlayObjectOptions } from "../infrastructure/Atomic.ts";
import { type RockSkyClientData, type RockSkyData, type RockSkyOptions } from "../infrastructure/config/client/rocksky.ts";
import type {AbstractApiOptions, FormatPlayObjectOptions} from "../infrastructure/Atomic.ts";
import type {RockSkyClientData, RockSkyData, RockSkyOptions} from "../infrastructure/config/client/rocksky.ts";
import AbstractApiClient from "./AbstractApiClient.ts";
import { isPortReachableConnect, joinedUrl, normalizeWebAddress } from '../../utils/NetworkUtils.ts';
import { type ListenResponse, type ListenType, type SubmitPayload } from '../../../core/vendor/listenbrainz/interfaces.ts';
import type {ListenResponse, ListenType, SubmitPayload} from '../../../core/vendor/listenbrainz/interfaces.ts';
import { playToListenPayload } from './listenbrainz/lzUtils.ts';
import { type RockskyScrobble } from './rocksky/interfaces.ts';
import { type Handle } from "@atcute/lexicons";
import type {RockskyScrobble} from './rocksky/interfaces.ts';
import type {Handle} from "@atcute/lexicons";
import { getATProtoIdentifier, identifierToAtProtoHandle } from './atproto/atUtils.ts';
import { baseFormatPlayObj } from "../../utils/PlayTransformUtils.ts";
import { ScrobbleSubmitError } from "../errors/MSErrors.ts";
import { tryApiCall } from "../../utils/RequestUtils.ts";
import { type CreateScrobbleInput, RockskyClient } from "@rocksky/sdk";
import { getRoot } from "../../ioc.ts";
import { MSCache } from "../Cache.ts";
import { type HandleData } from "../infrastructure/config/client/atproto.ts";
import type { MSCache } from "../Cache.ts";
import type {HandleData} from "../infrastructure/config/client/atproto.ts";
import { parseRegexSingle } from "@foxxmd/regex-buddy-core";
interface SubmitOptions {
+3 -3
View File
@@ -1,7 +1,7 @@
import { type AbstractApiOptions } from "../../infrastructure/Atomic.ts";
import { type TealClientData } from "../../infrastructure/config/client/tealfm.ts";
import type {AbstractApiOptions} from "../../infrastructure/Atomic.ts";
import type {TealClientData} from "../../infrastructure/config/client/tealfm.ts";
import { getATProtoIdentifier } from "./atUtils.ts";
import { type ATProtoAppData, type ATProtoUserIdentifierData } from "../../infrastructure/config/client/atproto.ts";
import type {ATProtoAppData, ATProtoUserIdentifierData} from "../../infrastructure/config/client/atproto.ts";
import { ATProtoAuthenticatedApiClient } from "./ATProtoAuthenticatedApiClient.ts";
import { PasswordSession, type PasswordSessionData } from '@atcute/password-session';
import { Client } from "@atcute/client";

Some files were not shown because too many files have changed in this diff Show More