* Use MS Player to mark Play as a repeat if session repeat is detected
* Remove FUZZY temporal accuracy as acceptable for detecting duplicate if candidate play is marked as repeat
* Add queue for aggregating Now Playing updates
* Filter play based on 1. source priority and 2. player stickiness or deterministic order
* Use toad scheduler to processing NP queue
* Define if scrobble client can update Now Playing based on class property, use this to filter now playing update
* Use thresholds to determine when/if Now Playing should be updated
* Set LastfmScrobbler as supported for Now Playing
* Allow non-positional Sources to still report position (for display only)
* Refactor Azuracast to be non-positional due to reported position not always being accurate (azuracast approximates track length)
* Rename from manualListening to Should Scrobble to better describe functionality
* Move check to Source scrobble function to avoid short circuting Source player for no reason
* Add to Azuracast and make functionality more generic
Clearing pre-stale listening range prevents any future resume (player becomes non-stale) from using old listening range to scrobble. Represents a physical "break" between listening sessions for the same song
Missing arg to pass to getPlayedObject to ensure the timestamp used for playDate is the same as normally discovered tracks. Without the arg two tracks played with the second being scrobbled on stale would cause the latter to have the same TS as the former
fixes#254
Replaces `build-push-action` buildx QEMU building for multiple `arch` with a multi-runner matrix that builds each image on native runners and combines digests before pushing to registries.
* Use arm public preview runner
* Refactor publish image workflow to use multi-runner
* Refactor PR workflow to use multi-runner
* Update to latest node 20.x minor version
* Remove `--verbose` and enable `--no-audit` for npm install to reduce install time
* These were left over from debugging ARM builds from Feb 2024
* Do play cleanup during polling for new plays so that we can discover tracks from abandoned players
* Fixes missed scrobbles from Plex API when Source stops returning a session due to player queue end #223
* Diff type should always be prepend since we are checking for new tracks
* Warn and log if another type is detected
* Warn if more than one track diffed for discovery
* Refactor and and make YTM history parsing testable
#227
* Fixes#241 by using the track artist as the primary artist when album artist is present
* Bug is reported in LukeHagar/plexjs#27 but will need be patched at runtime for the foreeable future due to npm packages not being published since September LukeHagar/plexjs#26
* try-catch and bubble up depending on usage
* replace poll/monitor code on startup with scheduler task usage
* better notification messages with truncated cause chain
* Notifications are not mission critical so its okay if they fail on init because service outage may be transient. Always init notification services and always attempt to push (we catch failures anyway)
* Parse more info from notification URL and test simple reachability of host/port to make troubleshooting easier
Fixes#215
* Patch generator to generate definition titles until vega/ts-json-schema-generator#2127 is resolved
* Add launch config for debugging schema generation
* Disallow topRef generation so docusarus json viewer works
* Write to docsite static assets on generation
Youtube TV seems no longer have scope for reading history or account details?
Cookie may work and custom oauth seems most stable based on reporting from LuanRT/YouTube.js#803
* Identify when session is for the local user (no user is logged in)
* Add Local User as default allowed when no usersAllow settings
* Allow user config to allow/block local user
* Update plex docs
Fixes#218
* Add session key dervied from source data to Play and PlayerState
* Discriminate states based on a session key, if possible
* Log which states are being dropped based on session key (plex)
* Add session key to important logging (new player play, in UI for player info)
* Introduce positional and non-positional data structures
* Refactor state player into types of each
* Refactor listen ranges and progress into types of each
* Implment (internal) real-time player and base positional player seeked/repeat on real-time drift instead of just reported position
* Refactor tests to use (emulated) real-time components
Plex only updates player position every 15 seconds (of played track) so player state needs to be adapted to not detect this as a pause or seek.
* Allow per-source drift allowed before triggering seek
* Allow per-source pause detection
Was not passing stopped state due to undefined play but need to do this to let last play actually be scrobbled.
Refactor jellyfin session validation and types for MemorySource to correctly allow processing player state with no plays
* Default disallow any activity not from a library of collectionType 'music'
* Allow user config for allow/block by library name
* Allow user config for allowing additional library collection types
* Update docs and example config
SDK types say AlbumArtists etc. should be defined but #200 shows that's not always the case, for some reason.
Check for undefined value on playback data that requires additional transformation to prevent errors.
Fixes#200
Fixes a lot of headaches:
* vega is more up to date and extensible
* no more hacky regex for reference names
* complete control over parsing and formatting (const as enum)
* does not duplicate typescript as dependency
* should properly throw errors on schema generation instead of just logging to console
* move schema static files into programmatic creation instead of package scripts
* fix types being removed in prod which caused schema generation errors
Not enough to include it in both dependencies and devDependencies -- it needs to be NOT in devDependencies to prevent `npm ci --omit=dev` from purging it.
* Add suite for testing client/source parsing and validation
* Additionally, use example configs for two-birds-one-stone checking they are valid
* Refactor sources init as separate step from component building
* Use runtime-generated schemas
* Ensures validated schemas always match code
* Can use schemas for individual source/clients without having to generate files
* catch invididual validation errors and log instead of crashing
* refactor validation error output into chained error instead of as individual log statements to improve clarity
* Swtch to devcontainer for node 18.x and remove terminal/launch settings for version (removes nvm dependency in container)
* Update eslint config to use proper config for include/ignore files
* Add spec for mocha and mocha test runner extension + settings
* Fix documentation and jellyfin example to match code
* Add tests for jellyfin api config parsing and valid activities
* Fix jellyin api ENV naming to match code
* Fix config parsing missing lowercase transform
Fixes#196
* All required files in the same file by using .actrc
* Include example .secrets and .env
* Update README with usage
* Refactor actTest to make docker build steps optional with ENV
* Add refreshMinInterval to prevent hammering upstream services
* Refactor refresh logic to be simpler and account for backlogged tracks
* (test): Improve play generation utils for testing
* (test): Refactor testing for upstream scrobble refreshing to use more actual scrobbler class behavior
* Move file parsing to own function
* Add some typings to file parsing to help with development (typings are bad)
* Defensively check structure of formdata for nested arrays since Plex seems to waffle on how it sends data (fixes#191)
* Add logging for all files found to help with future end-user debugging
express-static prefers directories over files when using the extensions
option, meaning an extensionless route will always route to the
directory instead of the file (expressjs/serve-static#138).
* Ensure pre-release tags don't also get latest tag
* Disable alpine builds
* Only increases testing and build complexity and there has been no feedback from users about needing it
* Aligns with "normal" default branch usage better, allows dependabot to actually run
* Change image publishing to push tagged images as 'latest' and master images as `edge`
* Implement new jellyfin source using JF typescript api client library for communication
* Use either API Key or username/password
* Implement real-time scrobble monitoring
* Implement feature parity filters from old jellyfin source (users) and add devices filters
* Refactor documentation for new configuration with api and add migration steps for webhook users
* Add deprecation warning to jellyfin webhook source
* Move recentScrobbles and generic refresh logic into abstract class
* New abstract function for scrobbler children to implement to get scrobbler-specific play lists on refresh
* Implement shouldRefereshScrobble with logging for better insight into why or why not a scrobbler refreshed upstream scrobbles
Since docs are now generated alongside builds and hosted on the app server there is no longer a barrier to using docusaurus docs as the main documentation presentation. Any user looking for docs can use the GH Pages hosted site or a "versioned" docs site hosted on their MS app. Due to this we can fully commit to docusaurus and take advantage of MDX layouts to simplify docs.
* Replace env/file configs headings in configuration with MDX tabs to make reading easier
* Remove superfluous TOCs
* Replace internal links in repo readme with links to docsite
* setup.sh takes care of building generated sources for project/docsite and building flatpak app
* Modify flatpak manifest to include two generated sources files and build docsite files
* Update flatpak docs with script usage and generated sources changes, move to docusauraus doc site
* Add npm scripts for installing/building docusaurus from main project
* Set base url based on ENV
* Build docsite in docker image
* Add link to doc in dashboard and link to github
* Update nodejs install instructions to include doc build command
* Update GH pages workflow to use correct base url
* Add hint page to main app when docs are not built
* Use lower headers so docusaurs sidebar TOC is more accurate
* Specify header levels for config/FAQ to make sidebar more concise
* Wrap inline TOC in details block -- so docusaurs users don't have to see it twice but its still available to github viewers
Sporadic DNS issues with alpine have caused a number of issues #160#126#134
Use debian as default since it has been proven to be stable. The image size increase is worth squashing these issues.
* Use superdiff to diff PlayObject lists and detect changes as well as append/prepend scenarios
* Replace YTM recently played logic with list diffing, only accept prepend-validated lists
* On non-prepend scenarios replace existing recently played and log human readable diff
* Patch youtube-music-ts-api to use updated cookies from response and provide a callback on update
* Implement currentCreds/build init data and read from MS-updated creds if available
* Write to currentCreds when ytm-ts-api invokes auth update callback and optionally log what parts changed based on config options
* Update Vite dep
* Use inlineViteConfig for vite-express middleware and detect correct base string
* Refactor vite.config.ts to detect correct base string
* Move code into testable function
* Fix port erasure when url uses standard port but it is explicitly specified
* Simplify building default url and fix using undefined mainPort variable
* Fix port erasure when https protocol is explicitly specified
* Refactor localUrl usage as URL instead of string
* Implement pathname joining on URL to make handling trailing/leading slashes take place in nodejs lib
* Implements tests for URL building/manipulation
* Increase default backlog limits to maximum pagination supported by each source
* Add scrobbleBacklogCount config property so user can explicitly set # of tracks to backlog
* Move some debug => verbose for non-repeating logs that are relevant to state, notifications, and scrobbling
* Move some verbose => debug that were repeating and making verbose level noisy
If a user does not need the dashboard or has security concerns about ingress and cannot control their hosting environment this enables reducing attack surface.
Closes#150
Error Cause (https://github.com/tc39/proposal-error-cause) was finalized and introduced in ES2022 so we don't need to ponyfill this anymore. Still using the helpful helper functions thought.
* Refactor SOT to be a type (may have more in the future)
* Set player stale/orphan interval based on SOT type
* Add note in log when initializing MemorySource which does not use player as SOT
* Add tooltip to UI player when it is not SOT
Migrating to @spotify/spotify-web-api-ts-sdk requires that the credentials we provide have valid expires_in/expires data and grant type. Make this migration easier by writing that data now so users upgrading later most likely do not need to re-authenticate.
* Actually remove device if it fails after X retries
* Properly close controllers/platform/client to prevent uncaught errors from castv2
* Move manually configured devices to device discovery function so they can be reconnected on heartbeat (if previously removed)
* Implement interfaces for response shape and getters for data
* Parse version data from response and use ping endpoint to return info on connection test
* Fix throwing auth/connection errors to include actual cause along with more human-friendly parsed error info
* Build error causes based on known error response types and expected maloja error body
* Only retry requests if error is network-related
* Set upstream error as NOT showstopping if we find expected error structure and status is less than 500
* Better maloja error parsing and logging
* Switch to lower-level mdns library for more control during debugging
* Use avahi-browse with host avahi-daemon to circumvent musl (alpline) issues with mdns resolution https://johnsiu.com/blog/alpine-linux/
* Add transportId shortname to logger label for better differentiation of "per session" apps
* Fix handling of MS player deletion when pruning apps by checking for other valid apps before deletion
* Refactor credit parsing into two stages: joiner separation and credits parsing
* Break credit parsing into wrapped vs. non-wrapped for simpler regexes
* Implement suffix matching after credits
* Add tests for wrapped vs. non-wrapped credits and with suffixes
* Break up initialize into build data => check connection => check auth
* Implement stub methods in AbstractSource with default return for steps
* Refactor existing initialize overrides into steps for each source
* Move (re)start to same location on status card as client and change verbage to be the same
* Fix api behavior to use correct poll function so that backlog is triggered
* Consolidate playerDelete event emitter to deletePlayer function
* Consolidate logging for player deletion to deletePlayer function
* Add removal reason when transferring player state for single-platform sources
The assumption is that single platform sources (like spotify) can only play from one platform at a time. If they are smart (like Spotify) the "new" platform picks up where the old platform left off WRT playing music. This commit implements the same behavior for MS player state by transferring existing state from the existing (previous) platform to the newly discovered platform.
* Track datetime player "completes" play at
* Add scrobble datetime SOC metadata
* Log SOC used when printing play datetime
* Use hinted SOC scrobble datetime when comparing existing scrobbles
* Use hinted SOC scrobble datetime when building scrobbler client scrobble payload
* Hint Spotify backlog plays use END play date for scrobble SOC
* Refactor using 'close' boolean to 'match' granularity
* Makes using granularity for future logic easier
* Easier logging for granularity in summary
* Remove intermediate temporal functions in classes for DRY and so we can use comparison results
* Add Time Detail to match breakdown for more visibility during logging
* Fix undefined token inclusion caused by token lists with differing lengths
* Always compare longer to shorter string so sameness parameter order is invariant
* Add comments to make logic easier to understand
* Add tests to test new functionality
* Move logging for init/auth into client instead of builder class
* Move startup from builder class to main backend and use initScrobbleMonitoring() instead of calling init/auth individually
Paves the way for manual restarting #114
If player is calculated as playing and tracks have positional data then start and end positions can be fudged to completion if jitter is small enough
* At beginning of new track if position is within 3 seconds of start of track fudge to 1 second
* At end of old track if position is within 3 seconds of end track fudge to completion
This can be achieved with high confidence due to smart polling near end of tracks added in 1d84f07a40
If player has an absolute position and play has duration we can determine if polling interval will occur after play would end. In this scenario we can shorten interval to before play ends to get more accurate data about play state close to end of track and report more accurate play time.
* Wake up timestamp was based on ts AFTER polling finished + interval. This means real-time interval between polling drifted by the amount of time it takes to execute polling loop. Fixed by getting ts at beginning of loop instead of at end.
* Implement interval function that can be overriding by child classes
* Format interval to make floats less ugly
* Implement abstract playToClientPayload function to make future testing easier
* Refactor play object to include album artist in data
* Refactor scrobble client payloads to include album artist, if applicable
* Discriminate between proper errors (permissions) and network errors
* Fix return value and playbackState flag set during post-auth hook
* Improve error logging using cause and upstream error
* Fix undefined destructure when device is not present in playingNow payload
* Implement type/guard for node network error
* Refactor auth in sources & clients to flag if error is network or api based
* Retry auth on heartbeat if auth failure was caused by network
* Implement initial tests for testing client auth
* Fix tests hanging due to running async functions
* Store logs by levels, fixes issue where rarer warn/info are pushed out of buffer by frequent debug/verbose
* Use FixedSizeList instead of homegrown buffer
* Implement API and ui controls for setting log limit
* Implement backend api endpoints for dead letter CRUD
* Include dead letter queue count in client status data
* Implement dead letter client view
* Partially working retry/remove actions for client (request is OK, RTK not working for update yet)
* Fix repeat end position percent test (should be 85% not 15%)
* Fix current play/range not being reset correctly on repeat
* Add reportedTS param to facilitate testing
* Use running/idle for scrobble client to indicate processing status
* Change polling sources status to polling/idle to better reflect behavior
* Add status indicator color to statuses
* Delay scrobble based on last scrobble attempt TS
* rename abstract scrobble func to doScrobble
* Implement scrobble() in abstract client and track last scrobble attempt time, delay based on arg or client property
* This way first scrobble isn't delayed and we only delay if subsequent scrobble attempts are close in time
* Make scrobble processing sleep time on empty queue configurable
* Implemented logic for processing and removing scrobbles from dead letter queue
* Only retry scrobbles under X retries based on config or argument
* Add removal-only function (for later use with UI)
* Move scrobble checking into client (out of client manager) for single responsibility
* Implement scrobble queue/deadletter
* Less chance of duplicate processing since sources can't scrobble concurrently
* Better throttling since multiple sources can't scrobble concurrently
* When scrobbling fails we can retry on delay just like polling and *keep* the failed scrobbles for later retry rather than dropping them
* Deadletter (to be fully implemented) for scrobbles that fail multiple times or are signalled as not upstream api issues by client
Wait until player gets a new play or becomes stale before trying to discover the (previous) play so that we store a more accurate amount of time the user actually listened to that play
When looking for dups and finding a fuzzy timestamp match, close title, and mismatch (but some matched) multiple artist -- we give a bonus to artist score since its likely a dup.
* Add text/* wildcard so any text-prefixed content-type is valid (Jellyfin should always be sending json anyway)
* Add logging WARN when body is empty
Fixes#101
* QoL improvement -- Instead of needing to rename a config file so it is not read by MS a user can now use the 'enable' property in all source/client configs to enable or disable loading the config
* Added property to all config examples
* Use lastfm 'nowPlaying' property in user listens for MS player implementation
* Do not use MS player as source of truth -- continue to use lfm listen history
Will not be used by listenbrainz source for actual scrobbling so make logs less noisy by suppressing what would normally be logging on newely added tracks from player
* Allows users to define a base URL to be used for default redirect URI generation
* Convenience mostly for docker users so they do not have to explicitly define redirect uri for each source/client
* Use in UI start up logging hints
files in node_module can have wierd ownership based on how a package owner published the package. If this occurs a rootless container build system could potentially have issues with subui/gid mapping for non-standard (high number) ids.
During image build recursively chown all node_modules files to normalize ownership and avoid this problem.
Fixes#96
* Due to increased scrobbled thresholds in 590610022d we need to pol more frequently to make sure more of the track is accurately monitored
* Decreased default from 30 to 10 seconds
* Decreased max interval from 60 to 30 seconds
* Better use of header
* Show name on separate line
* Only show client/source if ambiguous
* Move status into right side of header
* Reduce padding/margin on card grid
* Reduce padding on card headers and bodies
* Improve log experience
* Reduce log font size and convert to mono
* Use proper log message parsing to remove dangerous html set
* Check for stale/orphaned/dead in independent loop so they are not dependent on getting source updates to update
* Only emit events if player state has updated
* Create and implement api interface for player state data
* Serialize memory source players to object indexed by platform id
* Refactor event emitters to separate event data better from event metadata
* Update source slice when player update event is broadcast
Addresses #5 and #65 -- in default config use-cases where:
* request/polling retry attempts are low
* source was initially OK (auth OK, initial communication OK)
* and there is a non-auth related *upstream* communication issue
there is no fallback path for restarting source polling after upstream has normalized.
This feat implements a scheduled heartbeat task that attempts to restart polling every 20 minutes if this scenario is valid so that a good source config will recover without user intervention.
* Update flatpak build instructions to use build
* Add run command to readme
* Update default config to keep web port backwards compatible
* Fix flatpak application name in run command in installation docs
* Rename to backend to better reflect what it is
* Move server functionality out of entry file and into own server subdirectory
* Rename 'apis' folder to 'vendor' to reflect usage and move into common
* Rename 'clients' folder to 'scrobblers' reflect usage and prevent ambiguity with client CRA folder
* Changed logo color to be easy to view in both dark/light modes (better color in general, imo)
* Changed and cleaned up head/meta/manifest info and logos for docs and frontend
* If user tried to start polling while polling was already occuring it caused an infinite while loop
* Add more signals for stopping polling for a restart use case
* Result and criteria are returned from methods
* Logging method to summarize results
* Log results for jellyfin if using debug mode or explicit config option
* Will eventually migrate all non-source specific properties to this new property to simplify data config
* Add logPlayerState to allow configurable player state output to DEBUG logging
* Implements generic Player state tracking when using MemorySource
* State tracking will replace processRecentPlays and all its data structures
* Tracks reported player position OR real-time listened when position not available
* Records listened sessions for more exhaustive "listen to" duration and eventually comparing incoming data playedAt for dups
* Uses own tracked data to calculate most likely player state (stopped/started/paused) as well as health (stale/orphaned from no data)
* Tracks player's reported state, if provided
* Move UserDataSaved heuristics into scrobble method implementation with more descriptive logging
* Add more aggressive UserDataSaved filtering by discarding play if its found in memory tracking, regardless of play date
* Attempt to correct bad UTC offset for second UserDataSaved event
* Refactor candidate/discovered Play maps to use tuple id instead of string so user can be compared later
* Add option to check all discovered platform plays (since jellyfin UserDataSaved doesn't include device id in payload)
* Check for UserDataSaved notification with correct reason as well as sanity check last played date (since jellyfin will save as played regardless of duration played)
* Bypass play tracking
Docusaurus treats the files as assets and forces them to download when relative links. Instead, use external links to github so the user can stay in browser
For jellyfin/plex/tautulli -- `logFilterFailure` option in config allows controlling if and what log LEVEL user/server filter failures are logged. #85
* Updated schema
* Updated examples
* Also added expected/found detailed logging for filters to jellyfin #84
* Split user value on comma to align with what config describes as acceptable (comma-delim)
* Trim found values to fix any whitespace typos
* On disallowed event logging add expected/found values to help with debugging
Fixes#84
When a track contains multiple artist, only the first artist should be send to Last.fm since it doesn't support multiple artists per track.
Previous behaviour was to combine all artists in a single string.
Issue: #81
* Removes need to override poll() in child class and better defines when logic should be performed
* Fixes app crash when spotify credentials are present but invalid #76
Add a scrobbleThresholds object with 'duration' and 'percent' values to all sources and sourceDefaults. If both are defined then first passing condition trigger scrobble.
* duration, in seconds, tests how long MS has been tracking the play (since discovery)
* percent, only used if source provides track duration, tests time MS has been tracking play (since discovery) is some % of track duration
* Remove npm dev dependencies after app install/build
* Delete typescript files after build
* Provide a default config with file logging disabled and copy examples to config folder if empty
* Update winston-duplex to remove hard winston dependency
* Use scoped @foxxmd/winston instead of git commit
* Replace 'winston' with '@foxxmd/winston' in application
* Log warnings when any specific mpris info for a player isn't able to be parsed rather than failing entire poll attempt
* Fallback to undefined tracking progress if position cannot be parsed
* Add play object data "meta" with a brainz property and common mbid tags
* Parse MBID data from jellyfin webhook
* Add MBID data from play object to 'additional_info' when scrobbling to listenbrainz
Closes#72
label:Please check existing knowledge before opening an issue
options:
- label:I have [checked the FAQ](https://foxxmd.github.io/multi-scrobbler/docs/FAQ/) and [documentation](https://foxxmd.github.io/multi-scrobbler/docs/configuration/)
required:true
- label:I have [searched through existing issues](https://github.com/FoxxMD/multi-scrobbler/issues?q=sort%3Aupdated-desc) and [discussions.](https://github.com/FoxxMD/multi-scrobbler/discussions)
required:true
- type:markdown
attributes:
value:|
If this a how-do-I question, troublehooting, or generally **not** a bug/error with multi-scrobbler please consider [**starting a discussion**](https://github.com/FoxxMD/multi-scrobbler/discussions) instead of opening an issue.
- type:textarea
id:what-happened
attributes:
label:Describe the Bug
description:Describe what you expected to happen and what happened instead. Please provide as much detail as possible.
placeholder:The app should have scrobbled from Spotify but instead it crashed with an error about undefined properties, etc...
validations:
required:true
- type:dropdown
id:platform
attributes:
label:Platform
description:What platform are you running multi-scrobbler on?
options:
- Docker
- Local (NodeJS)
- Flatpak
default:0
validations:
required:true
- type:textarea
attributes:
label:Versions
description:What version of multi-scrobbler and issue-related sources/clients are running?
placeholder:|
- multi-scrobbler: 0.8.3 on docker
- maloja 3.1.4
- jellyfin 10.8.9
validations:
required:true
- type:textarea
id:logs
attributes:
label:Logs
description:|
Please copy and paste any/all relevant from the DEBUG level log output.
Even if your issue does not seem to be shown in the logs you should still add as much detail as possible, a _lack_ of issues in logs may be a good hint during debugging.
Your logs will be automatically formatted into code, do not need use backticks.
If possible reproduce the issue with [**Debug Mode** enabled](https://foxxmd.github.io/multi-scrobbler/docs/configuration#debug-mode).
description:Suggest a new feature or functionality
title:"feat: "
labels:["enhancement"]
body:
- type:checkboxes
attributes:
label:Please check existing knowledge before opening an issue
options:
- label:I have checked the [documentation](https://foxxmd.github.io/multi-scrobbler/docs/configuration/)
required:true
- label:I have [searched through existing issues](https://github.com/FoxxMD/multi-scrobbler/issues?q=sort%3Aupdated-desc) and [discussions.](https://github.com/FoxxMD/multi-scrobbler/discussions)
required:true
- type:textarea
attributes:
label:Describe the new functionality
description:Please describe the feature or functionality you want multi-scrobbler to have. Use as much detail as possible about the exact behavior you wish to see.
validations:
required:true
- type:textarea
attributes:
label:Configuration Example
description:|
If the functionality involves changes to configuration files please provide an example here
render:json5
- type:checkboxes
attributes:
label:Will you help implement it?
options:
- label:I am willing to contribute code towards completing this feature
- label:I am willing to help test preview builds for this feature
description:Request a new Source or Client to be implemented
title:"feat: "
labels:["enhancement"]
body:
- type:checkboxes
attributes:
label:Please check existing knowledge before opening an issue
options:
- label:I have checked the [documentation](https://foxxmd.github.io/multi-scrobbler/docs/configuration/)
required:true
- label:I have [searched through existing issues](https://github.com/FoxxMD/multi-scrobbler/issues?q=sort%3Aupdated-desc) and [discussions.](https://github.com/FoxxMD/multi-scrobbler/discussions)
required:true
- type:dropdown
attributes:
label:What Type?
description:Is this a Source or a Client?
options:
- Source
- Client
default:1
validations:
required:true
- type:input
attributes:
label:Name
description:"What is the name of the new service/software?"
placeholder:"My Cool Player"
validations:
required:true
- type:input
attributes:
label:Website
description:"Website for the service/software. Preferably a github repo or other source-code homepage"
validations:
required:true
- type:dropdown
attributes:
label:API Available?
description:Does the service have an API or other protocol that multi-scrobbler can use to communicate? **Without this the likelyhood of implementation is very low.**
options:
- 'Yes'
- 'No'
default:1
validations:
required:true
- type:dropdown
attributes:
label:Free To Use?
description:Is this service/software freely available or does it require purchasing/subscription?
options:
- 'Yes'
- 'No'
default:1
validations:
required:true
- type:dropdown
attributes:
label:Willing to finance development?
description:If the software is **non-free** are you will to provide a paid account/subscription to multi-scrobbler developer's in order to facilitate development/testing?
options:
- 'Yes'
- 'No'
validations:
required:false
- type:checkboxes
attributes:
label:Will you help implement it?
options:
- label:I am willing to contribute code towards completing this feature
- label:I am willing to help test preview builds for this feature
- type:textarea
attributes:
label:Additional Context
description:Add any other context about the new Source/Client here
Need to have credentials in a [`.secrets` file](https://nektosact.com/usage/index.html#secrets), copy and rename [`.secrets.example`](./secrets.example) to `.secrets`, then fill out blank fields. Required for docker/metadata-action to read...something. Fails with `Parameter token or opts.auth is required` if they are not supplied.
An ENV file can also be made by copy and renaming [`.env.example`](./env.example). Set `NO_DOCKER_BUILD=true` if you only want to test APP_VERSION and docker tags output.
If running a full docker build for multi-runner workflows you will need to create an [artifact server](https://github.com/nektos/act/issues/329#issuecomment-1905955589) for ACT to work:
# Based on https://docs.docker.com/build/ci/github-actions/multi-platform/#with-bake (https://github.com/docker/build-push-action/issues/671#issuecomment-1609106171)
# with example from here https://github.com/crazy-max/docker-linguist/blob/master/.github/workflows/build.yml that uses caching
#
# may have to use this to enable pushing to multiple registries? https://github.com/docker/build-push-action/discussions/1067#discussioncomment-8677682
"line":"[2024-12-26 01:58:48.261 -0500] DEBUG : [App] [Sources] [Jellyfin - unnamed] Last activity was at 01:58:48-05:00 | Next check in 10.00s | No new tracks discovered",
A javascript app to scrobble music you listened to, to [Maloja](https://github.com/krateng/maloja), [Last.fm](https://www.last.fm), and [ListenBrainz](https://listenbrainz.org)
* Supports scrobbling from many **Sources**
* [Spotify](/docs/configuration.md#spotify)
* [Plex](/docs/configuration.md#plex) or [Tautulli](/docs/configuration.md#tautulli)
* [Plex](https://foxxmd.github.io/multi-scrobbler/docs/configuration#plex) or [~~Tautulli~~](https://foxxmd.github.io/multi-scrobbler/docs/configuration#tautulli)
*Monitor status of Sources and Clients using [webhooks (Gotify or Ntfy)](/docs/configuration.md#webhook-configurations) or [healthcheck endpoint](/docs/configuration.md#health-endpoint)
* Monitor status of Sources and Clients using [webhooks (Gotify, Ntfy, Apprise)](https://foxxmd.github.io/multi-scrobbler/docs/configuration#webhook-configurations) or [healthcheck endpoint](https://foxxmd.github.io/multi-scrobbler/docs/configuration#health-endpoint)
* Supports configuring for single or multiple users (scrobbling for your friends and family!)
* Web server interface for stats, basic control, and detailed logs
* Graceful network and client failure handling (queued scrobbles that auto-retry)
* Smart handling of credentials (persistent, authorization through app)
* Easy configuration through ENVs or JSON
*Docker images for x86/ARM
*Modify data before scrobbling with [regular expression or search patterns](https://foxxmd.github.io/multi-scrobbler/docs/configuration/transforms)
* Install using [Docker images for x86/ARM](https://foxxmd.github.io/multi-scrobbler/docs/installation#docker) or [locally with NodeJS](https://foxxmd.github.io/multi-scrobbler/docs/installation#nodejs)
**Why should I use this over a browser extension and/or mobile app scrobbler?**
@@ -38,9 +62,7 @@ A javascript app to scrobble music you listened to, to [Maloja](https://github.c
**But I already scrobble my music to Last.fm/ListenBrainz, is multi-scrobbler for me?**
Yes! You can use [Last.fm as a **Source**](/docs/configuration.md#lastfm--source-) or [Listenbrainz as a **Source**](/docs/configuration.md#listenbrainz--source-) to forward scrobbles from your profile to any other Client! That way you can keep your current scrobble setup as-is but still get the benefit of capturing your data to a self-hosted location.
<img src="/assets/status-ui.jpg" width="800">
Yes! You can use [Last.fm as a **Source**](https://foxxmd.github.io/multi-scrobbler/docs/configuration#lastfm-source) or [Listenbrainz as a **Source**](https://foxxmd.github.io/multi-scrobbler/docs/configuration#listenbrainz-source) to forward scrobbles from your profile to any other Client! That way you can keep your current scrobble setup as-is but still get the benefit of capturing your data to a self-hosted location.
#- TZ=Etc/GMT # Specify timezone from TZ Database name found here https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
#- SPOTIFY_CLIENT_ID=
#- SPOTIFY_CLIENT_SECRET=
#- SPOTIFY_REDIRECT_URI=http://multi-scrobbler-host-IP:9078/callback ## Need to be whitelisted in Spotify dashboard and is used for creating the connection first time
#- MALOJA_URL=http://maloja:42010
#- MALOJA_API_KEY=
#- PUID=1000 # required if running docker on linux host, see main README Docker setup instructions
#- PGID=1000 # required if running docker on linux host, see main README Docker setup instructions
# set if using a source/client with redirect URI that you have not explicitly set and MS is NOT running on the same machine that you will view the dashboard from
# EX: You will view MS dashboard at 'http://192.168.0.101:9078' -> set BASE_URL=http://192.168.0.101:9078
#- BASE_URL=http://MyHostIP:9078
volumes:
- "./config:/config"
#networks:
# - (optional to add container to the same bridge network that maloja is inside to be able to use docker internal networking & dns to resolve and connect to maloja URL via http://maloja:port)
ports:
- 9078:9078# first port is the HOST port multi-scrobbler will serve UI and API on
- TZ=Etc/GMT# Specify timezone from TZ Database name found here https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
#- SPOTIFY_CLIENT_ID=
#- SPOTIFY_CLIENT_SECRET=
#- SPOTIFY_REDIRECT_URI=http://multi-scrobbler-host-IP:9078/callback ## Need to be whitelisted in Spotify dashboard and is used for creating the connection first time
#- MALOJA_URL=http://maloja:42010
#- MALOJA_API_KEY=
#- PUID=1000 # required if running docker on linux host, see main README Docker setup instructions
#- PGID=1000 # required if running docker on linux host, see main README Docker setup instructions
volumes:
- /path_on_host/multi-scrobbler-config:/config
#networks:
# - (optional to add container to the same bridge network that maloja is inside to be able to use docker internal networking & dns to resolve and connect to maloja URL via http://maloja:port)
ports:
- 9078:9078# first port is the HOST port multi-scrobbler will serve UI on
restart:unless-stopped
services:
multi-scrobbler:
image:foxxmd/multi-scrobbler
container_name:multi-scrobbler
environment:
- TZ=Etc/GMT# Specify timezone from TZ Database name found here https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
#- SPOTIFY_CLIENT_ID=
#- SPOTIFY_CLIENT_SECRET=
#- SPOTIFY_REDIRECT_URI=http://multi-scrobbler-host-IP:9078/callback ## Need to be whitelisted in Spotify dashboard and is used for creating the connection first time
#- MALOJA_URL=http://maloja:42010
#- MALOJA_API_KEY=
#- PUID=1000 # required if running docker on linux host, see main README Docker setup instructions
#- PGID=1000 # required if running docker on linux host, see main README Docker setup instructions
# set if using a source/client with redirect URI that you have not explicitly set and MS is NOT running on the same machine that you will view the dashboard from
# EX: You will view MS dashboard at 'http://192.168.0.101:9078' -> set BASE_URL=http://192.168.0.101:9078
#- BASE_URL=http://MyHostIP:9078
volumes:
- "./config:/config"
#networks:
# - (optional to add container to the same bridge network that maloja is inside to be able to use docker internal networking & dns to resolve and connect to maloja URL via http://maloja:port)
ports:
- 9078:9078# first port is the HOST port multi-scrobbler will serve UI and API on
* [Config could not be parsed](#config-could-not-be-parsed)
# Connection Issues
## Plex/Tautulli/Jellyfin don't connect
These three [sources](/README.md#source) are **ingress-based** which means that multi-scrobbler waits for the Plex/Tautulli/Jellyfin server to contact multi-scrobbler, as opposed to multi-scrobbler contacting the server.
multi-scrobbler will log information about any server that connects to it for these three services. In the logs it looks something like this:
```
2023-02-22T10:55:56-05:00 info : [Ingress - Plex ] Received request from a new remote address: ::ffff:192.168.0.140 (UA: PlexMediaServer/1.24.5.5173-8dcc73a59)
2023-02-22T10:55:56-05:00 info : [Ingress - Plex ] ::ffff:192.168.0.140 (UA: PlexMediaServer/1.24.5.5173-8dcc73a59) Received valid data from server examplePlex for the first time.
2023-02-22T10:55:56-05:00 warn : [Plex Request ] Received valid Plex webhook payload but no Plex sources are configured
```
It also logs if a server tries to connect to a URL that it does not recognize:
```
2023-02-22T11:16:12-05:00 debug : [App ] Server received POST request from ::ffff:192.168.0.140 (UA: PlexMediaServer/1.24.5.5173-8dcc73a59) to unknown route: /plkex
```
**So, if you do not see either of these in your logs then Plex/Tautulli/Jellyfin is not able to connect to your multi-scrobbler instance at all.**
This is not something multi-scrobbler can fix and means you have an issue in your network.
### Troubleshooting
Check or try all these steps before submitting an issue:
#### Turn on Debug Logging
First, turn on **debug** logging for multi-scrobbler by setting the environmental variable `LOG_LEVEL=debug`:
* using node `LOG_LEVEL=debug ... node src/index.js`
* using docker `docker run -e LOG_LEVEL=debug ... foxxmd/multi-scrobbler`
Check the output for any additional information.
#### Check Host name and URL
The URLs examples in the [configuration](/docs/configuration.md) documentation assume you are running Plex/Tautulli/Jellyfin on the same server as multi-scrobbler. If these are not the same machine then you need to determine the IP address or domain name that multi-scrobbler is reachable at and use that instead of `localhost` when configuring these sources. **This is likely the same host name that you would use to access the web interface for multi-scrobbler.**
EX `http://localhost:9078/plex` -> `http://192.168.0.140:9078/plex`
#### Check Firewall and Port Forwarding
If the machine multi-scrobbler is running on has a firewall ensure that port **9078** is open. Or if it is in another network entirely make sure your router is forwarding this port and it is open to the correct machine.
#### Check Source Service Logs
Plex/Tautulli/Jellyfin all have logs that will log if they cannot connect to multi-scrobbler. Check these for further information.
##### Plex
Settings -> Manage -> Console
##### Tautulli
Check the command-line output of the application or docker logs.
##### Jellyfin
Administration -> Dashboard -> Advanced -> Logs
## Jellyfin has warnings about undefined or missing data
Make sure you have
* [Configured the webhook plugin correctly](/docs/configuration.md#jellyfin)
* Checked the **Send All Properties(ignores template)** option in the webhook settings and **Saved**
multi-scrobbler is known to work on Jellyfin `10.8.9` with Webhook version `11.0.0.0`.
You can verify the payload sent from the webhook by modifying your jellyfin configuration to include `logPayload: true` which will output the raw payload to DEBUG level logging:
```json
[
{
"name":"MyJellyfin",
"clients":[],
"data":{
"users":["FoxxMD"],
"options":{
"logPayload":true
}
}
}
]
```
If your issue persists and you open an Issue for it please include the raw payload logs in your report.
## Spotify/Deezer/LastFM won't authenticate
Ensure any **client id** or **secrets** are correct in your configuration.
The callback/redirect URL for these services must be:
* the same address you would use to access the multi-scrobbler web interface
* the web-interface must be accessible from the browser you are completing authentication from.
If multi-scrobbler is not running on the same machine your browser is on then the default/example addresses (`http://localhost...`) **will not work.** You must determine the address you can reach the web interface at (such as `http://192.168.0.140:9078`) then use that in place of `localhost` in the callback URLs.
EX `http://localhost:9078/lastfm/callback` -> `http://192.168.0.220:9078/lastfm/callback`
# Configuration Issues
## Config could not be parsed
If you see something like this in your logs:
```
2023-02-19T10:05:42-06:00 warn : [App] App config file exists but could not be parsed!
2023-02-19T10:05:42-06:00 error : [App] Exited with uncaught error
2023-02-19T10:05:42-06:00 error : [App] Error: config.json could not be parsed
```
It means the JSON in your configuration file is not valid. Copy and paste your configuration into a site like [JSONLint](https://jsonlint.com/) to find out where errors you have and fix them.
[**Sources** and **Clients**](/README.md#how-does-multi-scrobbler-ms-work) are configured using environmental (ENV) variables and/or json files.
**MS will parse configuration from both configuration types.** You can mix and match configurations but it is generally better to stick to one or the other.
TIP: Check the [**FAQ**](/docs/FAQ.md) if you have any issues after configuration!
## ENV-Based Configuration
This is done by passing environmental variables and so does not require any files to run MS.
* Using a docker container EX `docker run -e "SPOTIFY_CLIENT_ID=yourId" -e "SPOTIFY_CLIENT_SECRET=yourSecret" ...`
* Using a local installations by exporting variables before running MS EX `SPOTIFY_CLIENT_ID=yourId SPOTIFY_CLIENT_SECRET=yourSecret node index.js`
Use ENV-based configuration if:
* You are the only person for whom MS is scrobbling for
* You have a very simple setup for MS such as one scrobble [Client](/README.md#client) and one [Source](/README.md#source) IE Plex -> Maloja
## File-Based Configuration
MS will parse configuration files located in the directory specified by the `CONFIG_DIR` environmental variable. This variable defaults to:
* Local installation -> `PROJECT_DIR/config`
* Docker -> `/config` (in the container) -- see the [install docs](/docs/installation.md#docker) for how to configure this correctly
Use File-based configuration if:
* You have many [Sources](/README.md#source)
* You have many of each type of **Source** you want to scrobble from IE 2x Plex accounts, 3x Spotify accounts, 1x
Funkwhale...
* You have more than one scrobble **Client** you want to scrobble to IE multiple Maloja servers
* You want only some **Sources** to scrobble to some **Clients** IE Fred's Spotify account scrobbles to Fred's Maloja
server, but not Mary's Maloja server
File-based configurations located in the `CONFIG_DIR` directory can be parsed from
* an **all-in-one** config file named `config.json` that contains information for all Sources and Clients and/or
* many **specific** files named based on the client/source to configure IE `plex.json``spotify.json`
There are **example configurations** for all Source/Client types and AIO config located in the [/config](/config) directory of this project. These can be used as-is by renaming them to `.json`.
For docker installations these examples are copied to your configuration directory on first-time use.
There is also a [**kitchensink example**](/docs/kitchensink.md) that provides examples of using all sources/clients in a complex configuration.
### All-in-One File Configuration
[**Explore the schema for this configuration, along with an example generator and validator, here**](https://json-schema.app/view/%23?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Faio.json)
Example directory structure:
```
/CONFIG_DIR
config.json
```
<details>
<summary>Config Example</summary>
```json5
// in config.json
{
//...
"sources":[
{
"name":"myConfig",
"type":"spotify",
"clients":[
"myMalojaClient"
],
"data":{
"clientId":"anExample"
//...
}
}
],
"clients":[
{
"name":"myFirstMalojaClient",
"type":"maloja",
"data":{
"url":"http://myMalojaServer.example",
// ...
}
}
]
}
```
</details>
`config.json` can also be used to set default behavior for all sources/clients using `sourceDefaults` and `clientDefaults` properties.
See [config.json.example](/config/config.json.example) for an annotated example or check out [the kitchen sink example](kitchensink.md).
### Specific File Configuration
Each file is named by the **type** of the Client/Source found in below sections. Each file as an **array** of that type of Client/Source.
Example directory structure:
```
/CONFIG_DIR
plex.json
spotify.json
maloja.json
```
<details>
<summary>Config Example</summary>
```json5
// in maloja.json
[
{
"name":"myFirstMalojaClient",
"data":{
"url":"http://myMalojaServer.example",
"apiKey":"myKey"
}
},
{
"name":"mySecondMalojaClient",
"data":{
"url":"http://my2ndMalojaServer.example",
"apiKey":"myKey"
}
}
]
```
</details>
See the [/config](/config) directory of this project for examples of each type of config file or reference specific files below.
# Source Configurations
## [Spotify](https://www.spotify.com)
To access your Spotify history you must [register an application](https://developer.spotify.com/dashboard) to get a
Client ID/Secret. Make sure to also whitelist your redirect URI in the application settings.
| `SPOTIFY_REDIRECT_URI` | No | `http://localhost:9078/callback` | URI must end in `callback` |
### File-Based
See [`spotify.json.example`](/config/spotify.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSpotifySourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Plex](https://plex.tv)
Check the [instructions](plex.md) on how to setup a [webhooks](https://support.plex.tv/articles/115002267687-webhooks) to scrobble your plays.
| `PLEX_USER` | No | | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
### File-Based
See [`plex.json.example`](/config/plex.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FPlexSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Tautulli](https://tautulli.com)
Check the [instructions](plex.md) on how to setup a notification agent.
| `TAUTULLI_USER` | No | | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
### File-Based
See [`tautulli.json.example`](/config/tautulli.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FTautulliSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Subsonic](http://www.subsonic.org/)
Can use this source for any application that implements the [Subsonic API](http://www.subsonic.org/pages/api.jsp) (such as [Airsonic](https://airsonic.github.io/))
**Known Issues:**
* "Time played at" is somewhat inaccurate since the api only reports "played X minutes ago" so...
* All scrobble times are therefore "on the minute" and you may experience occasional duplicate scrobbles
* "played X minutes ago" sometimes is also not reported correctly
* Multiple artists are reported as one value and cannot be separated
* If using [Airsonic Advanced](https://github.com/airsonic-advanced/airsonic-advanced) the password used (under **Credentials**) must be **Decodable**
| `SUBSONIC_URL` | Yes | | Base url of your subsonic-api server |
### File-Based
See [`subsonic.json.example`](/config/subsonic.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FSubSonicSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Jellyfin](https://jellyfin.org/)
Must be using Jellyfin 10.7 or greater
* In the Jellyfin desktop web UI Navigate to -> Administration -> Dashboard -> Plugins -> Catalog
* Under Notifications -> **Webhook** -> Install, then restart your server
* Navigate back to -> Administration -> Dashboard -> Plugins -> My Plugins -> Webhook
| `JELLYFIN_USER` | | | Comma-separated list of usernames (from Jellyfin) to scrobble for |
| `JELLYFIN_SERVER` | | | Comma-separated list of Jellyfin server names to scrobble from |
### File-Based
See [`jellyfin.json.example`](/config/jellyfin.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FJellySourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Last.fm (Source)](https://www.last.fm)
See the [Last.fm (Client)](#lastfm) setup for registration instructions. You may need to disable "Hide recent listening information" on your [privacy page](https://www.last.fm/settings/privacy) for this to work.
### ENV-Based
No support for ENV based for Last.fm as a client (only source)
### File-Based
See [`lastfm.json.example`](/config/lastfm.json.example), change `configureAs` to `source`. Or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FLastfmSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
You will need to run your own Listenbrainz server or have an account [on the official instance](https://listenbrainz.org/login/)
On your [profile page](https://listenbrainz.org/profile/) find your **User Token** to use in the configuration.
**NOTE:** You cannot use ENV variables shown in the [Listenbrainz Client config](#listenbrainz) -- multi-scrobbler assumes Listenbrainz ENVs are always used for the **client** configuration. You must use the file-based config from below to setup Listenbrainz as a Source.
### File-Based
See [`listenbrainz.json.example`](/config/listenbrainz.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FListenBrainzSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
**Change `configureAs` to `source`**
## [Deezer](https://deezer.com/)
Create a new application at [Deezer Developers](https://developers.deezer.com/myapps)
* Application Domain must be the same as your multi-scrobbler domain. Default is `localhost:9078`
* Redirect URL must end in `deezer/callback`
* Default would be `http://localhost:9078/deezer/callback`
After application creation you should have credentials displayed in the "My Apps" dashboard. You will need:
* **Application ID**
* **Secret Key**
* **Redirect URL** (if not the default)
**If no access token is provided...**
After starting multi-scrobbler with credentials in-place open the dashboard (`http://localhost:9078`) and find your Deezer source. Click **(Re)authenticate and (re)start polling** to start the login process. After login is complete polling will begin automatically.
| `DEEZER_CLIENT_ID` | Yes | | Your **Application ID** |
| `DEEZER_CLIENT_SECRET` | Yes | | Your **Secret Key** |
| `DEEZER_REDIRECT_URI` | No | `http://localhost:9078/deezer/callback` | URI must end in `deezer/callback` |
### File-Based
See [`deezer.json.example`](/config/deezer.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FDeezerSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Youtube Music](https://music.youtube.com)
Credentials for YT Music are obtained from a browser request to https://music.youtube.com **once you are logged in.** [Specific requirements are here and summarized below:](https://github.com/nickp10/youtube-music-ts-api/blob/master/DOCUMENTATION.md#authenticate)
* Open a new tab
* Open the developer tools (Ctrl-Shift-I) and select the “Network” tab
* Go to https://music.youtube.com and ensure you are logged in
Then...
1. Find and select an authenticated POST request. The simplest way is to filter by /browse using the search bar of the developer tools. If you don’t see the request, try scrolling down a bit or clicking on the library button in the top bar.
2.**Make sure **Headers** pane is selected and open
3. In the **Request Headers** section find and copy the **entire value** found after `Cookie:` and use this as the `cookie` value in your multi-scrobbler config
4. If present, in the **Request Headers** section find and copy the number found in `X-google-AuthUser` and use this as the value for `authUser` in your multi-scrobbler config
NOTES:
* YT Music authentication is "browser based" which means your credentials may expire after a (long?) period of time OR if you log out of https://music.youtube.com. In the event this happens just repeat the steps above to get new credentials.
* Communication to YT Music is **unofficial** and not supported or endorsed by Google. This means that **this integration may stop working at any time** if Google decides to change how YT Music works in the browser.
### File-Based
See [`ytmusic.json.example`](/config/ytmusic.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FYTMusicSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
MPRIS is a standard interface for communicating with Music Players on **linux operating systems.**
If you run Linux and have a notification tray that shows what media you are listening to, you likely have access to MPRIS.
multi-scrobbler can listen to this interface and scrobble tracks played by **any media player** that communicates to the operating system with MPRIS.
**NOTE:** multi-scrobbler needs to be running as a [**Local Installation**](/docs/installation.md#local) in order to use MPRIS. This cannot be used from docker.
| MPRIS_ENABLE | No | | Use MPRIS as a Source (useful when you don't need any other options) |
| MPRIS_BLACKLIST | No | | Comma-delimited list of player names not to scrobble from |
| MPRIS_WHITELIST | No | | Comma-delimited list of players names to ONLY scrobble from. Overrides blacklist |
### File-Based
See [`mpris.json.example`](/config/mpris.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FMPRISSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Mopidy](https://mopidy.com/)
Mopidy is a headless music server that supports playing music from many [standard and non-standard sources such as Pandora, Bandcamp, and Tunein.](https://mopidy.com/ext/)
multi-scrobbler can scrobble tracks played from any Mopidy backend source, regardless of where you listen to them.
### File-Based
See [`mopidy.json.example`](/config/mopidy.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FMopidySourceConfig/%23%2Fdefinitions%2FMopidyData?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
Configuration Options:
##### `url`
The URL used to connect to the Mopidy server. You MUST have [Mopidy-HTTP extension](https://mopidy.com/ext/http) enabled.
If no `url` is provided a default is used which assumes Mopidy is installed on the same server as multi-scrobbler: `ws://localhost:6680/mopidy/ws/`
Make sure the hostname and port number match what is found in the Mopidy configuration file `mopidy.conf`:
```
...
[http]
hostname = localhost
port = 6680
...
```
The URL used to connect ultimately must be formed like this: `[protocol]://[hostname]:[port]/[path]`
If any part of this URL is missing multi-scrobbler will use a default value, for your convenience. This also means that if any part of your URL is **not** standard you must explicitly define it.
Part => Default Value
* Protocol => `ws://`
* Hostname => `localhost`
* Port => `6680`
* Path => `/mopidy/ws/`
<details>
<summary>URL Transform Examples</summary>
```json
{
"url":"mopidy.mydomain.com"
}
```
MS transforms this to: `ws://mopidy.mydomain.com:6680/mopidy/ws/`
```json
{
"url":"192.168.0.101:3456"
}
```
MS transforms this to: `ws://192.168.0.101:3456/mopidy/ws/`
```json
{
"url":"mopidy.mydomain.com:80/MOPWS"
}
```
MS transforms this to: `ws://mopidy.mydomain.com:80/MOPWS`
</details>
#### URI Blacklist/Whitelist
If you wish to disallow or only allow scrobbling from some sources played through Mopidy you can specify these using `uriBlacklist` or `uriWhitelist` in your config. multi-scrobbler will check the list to see if any string matches the START of the `uri` on a track. If whitelist is used then blacklist is ignored. All strings are case-insensitive.
EX:
```json
{
"uriBlacklist":["soundcloud"]
}
```
Will prevent multi-scrobbler from scrobbling any Mopidy track that start with a `uri` like `soundcloud:song:MySong-1234`
#### Album Blacklist
For certain sources (Soundcloud) Mopidy does not have all track info (Album) and will instead use "Soundcloud" as the Album name. You can prevent multi-scrobbler from using this bad Album data by adding the fake Album name to this list. Multi-scrobbler will still scrobble the track, just without the bad data. All strings are case-insensitive.
EX:
```json
{
"albumBlacklist":["SoundCloud","Mixcloud"]
}
```
If a track would be scrobbled like `Album: Soundcloud, Track: My Cool Track, Artist: A Cool Artist`
then multi-scrobbler will instead scrobble `Track: My Cool Track, Artist: A Cool Artist`
## [JRiver](https://jriver.com/)
In order for multi-scrobbler to communicate with JRiver you must have [Web Server Interface](https://wiki.jriver.com/index.php/Web_Service_Interface#Documentation_of_Functions) enabled. This can can be in the JRiver GUI:
* Tools -> Options -> Media Network
* Check `Use Media Network to share this library...`
* If you have `Authentication` checked you will need to provide the **Username** and **Password** in the ENV/File configuration below.
#### URL
If you do not provide a URL then a default is used which assumes JRiver is installed on the same server as multi-scrobbler: `http://localhost:52199/MCWS/v1/`
* Make sure the port number matches what is found in `Advanced` section in the [Media Network](#jriver) options.
* If your installation is on the same machine but you cannot connect using `localhost` try `0.0.0.0` instead.
The URL used to connect ultimately must be formed like this: `[protocol]://[hostname]:[port]/[path]`
If any part of this URL is missing multi-scrobbler will use a default value, for your convenience. This also means that if any part of your URL is **not** standard you must explicitly define it.
Part => Default Value
* Protocol => `http://`
* Hostname => `localhost`
* Port => `52199`
* Path => `/MCWS/v1/`
<details>
<summary>URL Transform Examples</summary>
```json
{
"url":"jriver.mydomain.com"
}
```
MS transforms this to: `http://jriver.mydomain.com:52199/MCWS/v1/`
```json
{
"url":"192.168.0.101:3456"
}
```
MS transforms this to: `http://192.168.0.101:3456/MCWS/v1/`
```json
{
"url":"mydomain.com:80/jriverReverse/MCWS/v1/"
}
```
MS transforms this to: `http://mydomain.com:80/jriverReverse/MCWS/v1/`
| JRIVER_URL | Yes | http://localhost:52199/MCWS/v1/ | The URL of the JRiver server |
| JRIVER_USERNAME | No | | If authentication is enabled, the username set |
| JRIVER_PASSWORD | No | | If authenticated is enabled, the password set |
### File-Based
See [`jriver.json.example`](/config/jriver.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FJRiverSourceConfig/%23%2Fdefinitions%2FJRiverData?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
| `MALOJA_URL` | Yes | | Base URL of your installation |
| `MALOJA_API_KEY` | Yes | | Api Key |
### File-Based
See [`maloja.json.example`](/config/maloja.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FMalojaClientConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fclient.json)
## [Last.fm](https://www.last.fm)
[Register for an API account here.](https://www.last.fm/api/account/create)
The Callback URL is actually specified by multi-scrobbler but to keep things consistent you should use
```
http://localhost:9078/lastfm/callback
```
or replace `localhost:9078` with your own base URL
| `LASTFM_API_KEY` | Yes | | Api Key from your API Account |
| `LASTFM_SECRET` | Yes | | Shared secret from your API Account |
| `LASTFM_REDIRECT_URI` | No | `http://localhost:9078/lastfm/callback` | Url to use for authentication. Must include `lastfm/callback` somewhere in it |
| `LASTFM_SESSION` | No | | Session id. Will be generated by authentication flow if not provided. |
### File-Based
See [`lastfm.json.example`](/config/lastfm.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FLastfmClientConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fclient.json)
## [Listenbrainz](https://listenbrainz.org)
You will need to run your own Listenbrainz server or have an account [on the official instance](https://listenbrainz.org/login/)
On your [profile page](https://listenbrainz.org/profile/) find your **User Token** to use in the configuration.
| LZ_TOKEN | Yes | | User token from your LZ profile |
| LZ_USER | Yes | | Your LZ username |
| LZ_URL | No | https://api.listenbrainz.org/ | The base URL for the LZ server |
### File-Based
See [`listenbrainz.json.example`](/config/listenbrainz.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FListenBrainzClientConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fclient.json)
# Monitoring
multi-scrobbler supports some common webhooks and a healthcheck endpoint in order to monitor Sources and Clients for errors.
## Webhook Configurations
Webhooks will **push** a notification to your configured servers on these events:
* Source polling started
* Source polling retry
* Source polling stopped on error
* Scrobble client scrobble failure
Webhooks are configured in the main [config.json](#all-in-one-file-configuration) file under the `webhook` top-level property. Multiple webhooks may be configured for each webhook type. EX:
```json
{
"sources":[
...
],
"clients":[
...
],
"webhooks":[
{
"name":"FirstGotifyServer",
"type":"gotify",
"url":"http://192.168.0.100:8070",
"token":"abcd"
},
{
"name":"SecondGotifyServer",
"type":"gotify",
...
},
{
"name":"NtfyServerOne",
"type":"ntfy",
...
},
...
]
}
```
### [Gotify](https://gotify.net/)
Refer to the [config schema for GotifyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FGotifyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info``warn` and `error` mappings.
EX
```json
{
"type":"gotify",
"name":"MyGotifyFriendlyNameForLogs",
"url":"http://192.168.0.100:8070",
"token":"AQZI58fA.rfSZbm",
"priorities":{
"info":5,
"warn":7,
"error":10
}
}
```
### [Ntfy](https://ntfy.sh/)
Refer to the [config schema for NtfyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FNtfyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info``warn` and `error` mappings.
EX
```json
{
"type":"ntfy",
"name":"MyNtfyFriendlyNameForLogs",
"url":"http://192.168.0.100:9991",
"topic":"RvOwKJ1XtIVMXGLR",
"username":"Optional",
"password":"Optional",
"priorities":{
"info":3,
"warn":4,
"error":5
}
}
```
## Health Endpoint
An endpoint for monitoring the health of sources/clients is available at GET `http://YourMultiScrobblerDomain/health`
* Returns `200 OK` when **everything** is working or `500 Internal Server Error` if **anything** is not
* The plain url (`/health`) aggregates status of **all clients/sources** -- so any failing client/source will make status return 500
* Use query params `type` or `name` to restrict client/sources aggregated IE `/health?type=spotify` or `/health?name=MyMaloja`
* On 500 the response returns a JSON payload with `messages` array that describes any issues
* For any clients/sources that require authentication `/health` will return 500 if they are **not authenticated**
* For sources that poll (spotify, yt music, subsonic) `/health` will 500 if they are **not polling**
* (Optionally) map the web UI port in the container **9078** to the host
* With [docker](https://docs.docker.com/engine/reference/commandline/run/#publish): `-p 9078:9078` (first port is the port on the host to use)
* With [docker-compose](https://docs.docker.com/compose/compose-file/compose-file-v3/#short-syntax-1): `- "9078:9078"`
* (Optionally) set the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) for the container using the environmental variable `TZ` ([docker](https://docs.docker.com/engine/reference/commandline/run/#env)) ([docker-compose](https://docs.docker.com/compose/compose-file/compose-file-v3/#environment))
### Linux Host
If you are
* using [rootless containers with Podman](https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics#why_podman_)
* running docker on MacOS or Windows
this **DOES NOT** apply to you.
If you are running Docker on a **Linux Host** you must specify `user:group` permissions of the user who owns the **configuration directory** on the host to avoid [docker file permission problems.](https://ikriv.com/blog/?p=4698) These can be specified using the [environmental variables **PUID** and **PGID**.](https://docs.linuxserver.io/general/understanding-puid-and-pgid)
To get the UID and GID for the current user run these commands from a terminal:
*`id -u` -- prints UID
*`id -g` -- prints GID
### Docker Usage Examples
#### Using [env-based](/docs/configuration.md#env-based-configuration) configuration
These sources are **ingress-based** which means that multi-scrobbler waits for the Webscrobbler extension or app using Listenbrainz Endpoint to contact multi-scrobbler, as opposed to multi-scrobbler contacting them.
multi-scrobbler will log information about any server that connects to it for these services. In the logs it looks something like this:
```
[2025-07-10 12:32:21.417 -0400] INFO : [App] [API] [Ingress] [WebScrobbler] [Ingress] [WebScrobbler] Received request from a new remote address: ::ffff:127.0.0.1 (192.168.0.100) (UA: Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0)
[2025-07-10 12:32:21.418 -0400] INFO : [App] [API] [Ingress] [WebScrobbler] [Ingress] [WebScrobbler] ::ffff:127.0.0.1 (192.168.0.100) (UA: Mozilla/5.0 (X11; Linux x86_64; rv:139.0) Gecko/20100101 Firefox/139.0) Received data for API URL slug '(none)' for the first time.
```
It also logs if a server tries to connect to a URL that it does not recognize:
```
2023-02-22T11:16:12-05:00 debug : [App ] Server received POST request from ::ffff:192.168.0.100 (UA: PlexMediaServer/1.24.5.5173-8dcc73a59) to unknown route: /plkex
```
**So, if you do not see either of these in your logs then the Source is not able to connect to your multi-scrobbler instance at all.**
This is not something multi-scrobbler can fix and means you have an issue in your network.
#### Troubleshooting
Check or try all these steps before submitting an issue:
##### Turn on Debug Logging
First, turn on **debug** logging for multi-scrobbler by setting the environmental variable `DEBUG_MODE=true`. Then, check the output for any additional information.
##### Check Host name and URL
The URLs examples in the [configuration](configuration/configuration.mdx) documentation assume you are running Webscrobbler/LFM Endpoint on the same server as multi-scrobbler. If these are not the same machine then you need to determine the IP address or domain name that multi-scrobbler is reachable at and use that instead of `localhost` when configuring these sources. **This is likely the same host name that you would use to access the web interface for multi-scrobbler.**
EX `http://localhost:9078/api/webscrobbler` -> `http://192.168.0.140:9078/api/webscrobbler`
##### Check Firewall and Port Forwarding
If the machine multi-scrobbler is running on has a firewall ensure that port **9078** is open. Or if it is in another network entirely make sure your router is forwarding this port and it is open to the correct machine.
##### Check Source Service Logs
###### Webscrobbler
See [Debugging the extension](https://github.com/web-scrobbler/web-scrobbler/wiki/Debug-the-extension) to get logs which should have information about failed requests.
### Spotify/Deezer/LastFM won't authenticate
Ensure any **client id** or **secrets** are correct in your configuration.
The callback/redirect URL for these services must be:
* the same address you would use to access the multi-scrobbler web interface
* the web-interface must be accessible from the browser you are completing authentication from.
If multi-scrobbler is not running on the same machine your browser is on then the default/example addresses (`http://localhost...`) **will not work.** You must determine the address you can reach the web interface at (such as `http://192.168.0.140:9078`) then use that in place of `localhost` in the callback URLs.
EX `http://localhost:9078/lastfm/callback` -> `http://192.168.0.220:9078/lastfm/callback`
### Youtube Music fails after some time
If you see errors in multi-scrobbler for YTM that contain **401** or **403** like
```
Error: Could not send the specified request to browse. Status code: 401
```
then YTM has invalidated your authentication.
First, ensure you are NOT using [YoutubeTV authentication.](configuration/configuration.mdx?ytmAuth=ytt#youtube-music) If you completed authentication by entering a "User Code" you are using YoutubeTV which has stopped working. You should reauthenticate using **Cookies** or **Custom OAuth.**
#### When using OAuth Client Authentication
Refresh your authentication by using the **(Re)authenticate** link from MS's web dashboard.
#### When using Cookies Authentication
The library MS uses relies on scraping the YTM site by using cookies from your actual browser to pretend it is a browser. It does its best to keep these up to date but since this is not an official way to access the service YTM may invalidate your access _to the authenticated session_ at any time. How this is triggered is unknown and not something multi-scrobbler can control. You can help limit the chance of your session being invalidated by [getting the cookie from an Incognito/Private Session](https://github.com/LuanRT/YouTube.js/issues/803#issuecomment-2504032666) and then immediately closing the browser afterwards.
To re-authenticate MS [follow the YTM instructions to retrieve a new set of cookies for multi-scrobbler](configuration/configuration.mdx?ytmAuth=cookie#youtube-music) and then restart MS to potentially resolve the problem.
## Configuration Issues
### Config could not be parsed
If you see something like this in your logs:
```
2023-02-19T10:05:42-06:00 warn : [App] App config file exists but could not be parsed!
2023-02-19T10:05:42-06:00 error : [App] Exited with uncaught error
2023-02-19T10:05:42-06:00 error : [App] Error: config.json could not be parsed
```
It means the JSON in your configuration file is not valid. Copy and paste your configuration into a site like [JSONLint](https://jsonlint.com/) to find out where errors you have and fix them.
## Scrobbling Issues
### Last.fm does not scrobble tracks with multiple artists correctly
This is a limitation of the [Last.fm API](https://www.last.fm/api/show/track.scrobble) where the **artist** field is only one string and Last.fm does not recognize (play well) with "combined" artists.
Multi-scrobbler works the same was the official Spotify-Last.fm integration works -- it only scrobbles the **first** artist on a multi-artist track.
### Google Cast track information is missing/incorrect or MS player has weird times
The Google Cast integration relies on a few common fields in the data it receives from your casting device. Every platform that can cast (Spotify, Pandora, etc...) *should* use these fields the same but there are slight differences between their implementations that may confuse multi-scrobbler. Specific platforms may also return more information in non-common fields that are undocumented.
To diagnose these issues you [**must enable payload logging**](configuration/configuration.mdx#cast-troubleshooting) for your google cast Source, run MS, and then include logs with this output from that run. Without the raw data logged from your cast device it will be nearly impossible to resolve your issue.
### Google Cast device does not track media
It is likely the app playing on the cast device is incorrectly reporting the media type as **not music**.
MS logs will tell you what type the media is reported as with lines like:
```
My Artist - Example Track has 'unknown' media type and allowUnknownMedia=false, will not track
```
Refer to [Allow Unknown Media Type](configuration/configuration.mdx#allow-unknown-media-type) section to fix this
```
My Artist - Example Track has 'movie' media type so will not track
```
Refer to [Force Media Tracking](configuration/configuration.mdx#forcing-media-tracking) section to fix this
### VLC is not scrobbling fields correctly
Before reporting an issue turn on metadata logging in the MS VLC configuration, [see the VLC documentation.](configuration/configuration.mdx#vlc-information-reporting)
### Youtube Music misses or duplicates scrobbles
<details>
<summary>How MS detects YTM history</summary>
In order for multi-scrobbler to accurately determine if a song has been scrobbled it needs **a source of truth.** For YTM this is a "history" list scraped from the YTM website. Unfortunately, the data in this list can be (often) inconsistent which makes it hard for multi-scrobbler to "trust" that it is correct and determine when/if new scrobbles occur. This inconsistency is not something multi-scrobbler can control -- it is a side-effect of having to use an unofficial method to access YTM (scraping).
To compensate for this multi-scrobbler resets when it considers this list the "source of truth" based on if the list changes in an inconsistent way between consecutive checks. New scrobbles can only be detected when this list is "OK" as a source of truth for N+1 checks. Therefore, any new tracks that appear when the list is inconsistent will be ignored.
Duplicate scrobbles can also occur if the change between two checks is technically consistent. For instance, if you listen to a track twice in some period, separated by other music, YTM will sometimes "remove" the track from the earlier time (further down in your history) and "re-add" it at the top of the history.
</details>
#### Reporting YTM scrobble issues
If you experience scrobble detection issues you can help improve MS's YTM heureistic by providing thorough feedback as [an issue.](https://github.com/FoxxMD/multi-scrobbler/issues/new?assignees=&labels=bug&projects=&template=01-bug-report.yml&title=bug%3A+) **Please do the following to provide the most useful report:**
##### Turn on Change Detection
In your YTM configuration (`ytmusic.json`) add `logDiff` under `options` like this:
This will cause MS to log YTM history changes similar to this:
```
[Ytmusic - MyYTM] Changes from last seen list:
1. (tuhe1CpHRxY) KNOWER - I’m The President --- undefined => Moved - Originally at 6
2. (Mtg8V6Xa2nc) Vulfpeck - Romanian Drinking Song --- Schvitz => Moved - Originally at 1
3. (rxbCaiyYSXM) Nightmares On Wax - You Wish --- In A Space Outta Sound => Moved - Originally at 2
4. (tMt_YXr90AM) Gorillaz - O Green World --- undefined => Moved - Originally at 3
...
```
Which are essential to troubleshooting this behavior.
##### Turn on File Logging
Optionally, if your logs are busy due to many other sources, you can log **only** YTM logs to a file. Turn this on with the `logToFile` config option:
```json
{
"type":"ytmusic",
"name":"MyYTM",
"data":{...},
"options":{
"logDiff":true,
"logToFile":true
}
}
```
This will cause MS to write YTM logs to a file in your `logs` folder named `ytmusic-MyYTM.log` (based on your source name).
##### Provide Detail and Context
Provide a detailed account of how you were using YTM when the issue occurred, including things like:
* the platform listening on (desktop, mobile, 3rd party client, etc...)
* any changes in platform
* > I switched from desktop to listening on my phone...
* how you were listening to music
* > I was playing an album start to finish
* > I listened to two songs in a row, then browsed for a new song in library by artist, then went back to a song in the queue...
Explain the expected behavior (it should have scrobbled songs x, y, then z) and what actually happened (it scrobbled songs x, then y, then x again, then z)
Provide ALL logs from the time when the issue occurred including logs from BEFORE (ideally 2-3 minutes of logs) and AFTER the issue.
Multi-scrobbler >= 0.8.8 supports a Plex Source that [directly uses the API](configuration/configuration.mdx?plexType=api#plex) which removes the need to use Tautulli since it does not require Plex Pass.
The Tautulli source **and** Plex Webhook source are deprecated and will be removed in an upcoming release. Existing users should migrate to the API source as soon as possible.
:::
Tracks played on [Plex](https://plex.tv/) can be scrobbled either by:
* A [Tautulli](https://tautulli.com/) notification agent with a webhook.
* Using Plex [Webhooks](https://support.plex.tv/articles/115002267687-webhooks) (restricted to Plex Pass users)
Multi-scrobbler configs support the ability to modify scrobble data in an automated fashion by matching and replacing strings in **title, artists, and album** at many different times in multi-scrobbler's lifecycle.
### Why?
You may need to "clean up" data from a Source or before sending to a scrobble Client due to any number of reasons:
* ID3 tags in your music collection are dirty or have repeating garbage IE `[YourMusicSource.com] My Artist - My Title`
* A Source's service often incorrectly adds data to some field IE `My Artist - My Title (Album Version)` when the title should just be `My Title`
* An Artist you listen to often is spelled different between a Source and a Client which causes duplicate scrobbles
In any scenario where a repeating pattern can be found in the data it would be nice to be able to fix it before the data gets downstream or to help prevent duplicate scrobbling. Multi-scrobbler can help you do this.
## Overview
### Journey of a Scrobble
First, let's recap the lifecycle of a scrobble in multi-scrobbler:
**Sources** are the beginning of the journey for a **Play** (song you've listened to long enough to be scrobblable)
* A Source finds a new valid **Play**
* The Source **compares** this new Play to all the other Plays it has already seen, if the Play is unique (title/artist/album/listened datetime) then...
* The Source **discovers** the Play, adds it to Plays it has seen already, and broadcasts the Play should be scrobbled to all Clients
Scrobble **Clients** listen for discovered Plays from Sources, then...
* A Client receives a **Play** from a Source
* The Client **compares** this Play to all the other scrobbles it has already seen, if the Play is unique (title/artist/album/listened datetime) then...
* The Client **scrobbles** the Play downstream to the scrobble service and adds it as a Scrobble it has seen already
### Lifecyle Hooks
You'll notice there is a pattern above that looks like this:
* **Before** data is compared
* Data is **compared**
* **After** data is compared
These points, during both Source and Client processes, are when you can hook into the scrobble lifecycle and modify it.
#### TLDR
In more concrete terms this is the structure of hooks within a configuration (can be used in any **Source** or **Client**):
```json5 title="lastfm.json" {10-14}
[
{
"name": "myLastFm",
"enable": true,
"configureAs": "source",
"data": {
// ...
},
"options": {
"playTransform": {
"preCompare": {/* ... */},
"compare": {/* ... */},
"postCompare": {/* ... */}
}
}
}
]
```
##### Hook
For **Sources**:
* `preCompare` - modify Play data immediately when received
* `compare` - temporarily modify Play data when it is being compared to see if Play was already discovered
* `postCompare` - modify Play data before sending to scrobble **Clients**
For **Clients**:
* `preCompare` - modify Play data immediately when received
* `compare` - temporarily modify Play data when it is being compared to see if it was already scrobbled
* `postCompare` - modify Play data before scrobbling it to downstream service and adding to already seen scrobbles
:::tip
Keep in mind that modifying Scrobble/Play data earlier in the lifecycle will affect that data at all times later in the lifecycle.
For example, to modify the track so it's the same anywhere it is processed in multi-scrobbler you only need to modify it in the **Source's** `preCompare` hook because all later processes will receive the data with the modified track.
:::
### Modification Parts
Each [**hook**](#hook) (`preCompare` etc...) is an object that specifies what part of the **Play** to modify:
```json5
{
"title": [/* ... */],
"artists": [/* ... */],
"album": [/* ... */]
}
```
##### Expression
and then a **list** what pattern/replacements (expressions) to use for the modification by using either simple strings or `search-replace` objects:
```json5
[
"badTerm", // remove all instances of 'badTerm'
{
"search": "anotherBadTerm", // and also match all instances of 'anotherBadTerm'
"replace": "goodTerm" // replace with the string 'goodTerm'
}
]
```
Putting it all together:
```json5 title="lastfm.json"
[
{
"name": "myLastFm",
"enable": true,
"configureAs": "source",
"data": {
// ...
},
"options": {
"playTransform": {
"preCompare": {
"title": [
"badTerm",
{
"search": "badTerm",
"replace": "goodTerm"
}
]
},
}
}
}
]
```
:::note
If the value of the field (title, an artist, album) is an empty string after transforming then the field is **removed.**
:::
:::tip
Modifications can also be applied to **all Sources** or **all Clients** when using the [AIO Config](./configuration.mdx?configType=aio#configuration-types) `config.json` by setting `playTransform` in `sourceDefaults` or `clientDefaults`:
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"sourceDefaults": { // will apply playTransform to all sources
"playTransform": {
"preCompare": {
"title": [
"(Album Version)"
]
}
}
},
"sources": [/* ... */],
"clients": [/* ... */]
}
```
</details>
:::
#### Compare Hook
The `compare` [hook](#hook) is slightly different than `preCompare` and `postCompare`. It consists of an object where you define which side(s) of the comparison should be modified. It also **does not modify downstream data!** Instead, the modifications are made only for use in the comparison.
```json5 title="lastfm.json"
[
{
"name": "myLastFm",
// ...
"options": {
"playTransform": {
"compare": {
"candidate": {/* ... */}, // modify the "new" Play being compared
"existing": {/* ... */}, // modify all "existing" Play/Scrobbles the new Play is being compared against
},
}
}
}
]
```
#### Regular Expressions
In addition to plain strings [expressions](#expression) that are matched and removed you can also use Regular Expressions. Write your regex like you normally would, but as a string, and it'll automatically be parsed:
```json5
[
"/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)' and removes it
{
"search": "/^\(\w+.com)/i", // matches any string that starts with '(YourMusic.com)'
"replace": "[MySite.com]" // replace with the string '[MySite.com]'
}
]
```
The `replace` property uses javascript's [`replace()` function and so can use any special string characters.](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#specifying_a_string_as_the_replacement)
### Conditional Modification
#### "When" Condition
Top-level hooks **and** individual rules also support a `when` key for testing **if they should be run.**
The `when` key is similar to a normal [modification](#modification-parts) except:
* the keys accept a single string instead of an array
* the `when` key data is an array instead of a single object
All parts of an individual `when` clause must test true to "pass" but if **any** `when` clauses pass the hook/rule is processed. Example `when` data:
```json5
{
"when": [
{
"artist": "Elephant Gym", // both of these must match the Play object (AND)
"album": "Dreams" // both of these must match the Play object (AND)
},
// OR
{
"title": "/(Remastered)$/", // both of these must match the Play object (AND)
"album": "Various Artists" // both of these must match the Play object (AND)
}
]
}
```
More succinctly:
* All parts (`artist` `album` `title`) of a `when` are `AND` conditions
* All part-objects in the `when` array are `OR` conditions
<details>
<summary>Example of top-level hook with when condition</summary>
```json5
{
// IF the artist is Elephant Gym
// THEN Run preCompare hook ELSE skip this hook
//
// Run search-replace on album
// Run regex title remove
"sourceDefaults": {
"playTransform": {
"preCompare": {
"when": [
{
"artist": "/Elephant Gym/"
}
],
"album": [
{
"search": "Dreams",
"replace": "夢境"
}
],
"title": ["/\s\-\s滾石40\s滾石撞樂隊\s40團拚經典(.+)$/i"]
},
}
}
}
```
</details>
<details>
<summary>Example of individual rule with when condition</summary>
```json5
{
// Always run preCompare
//
// On search-replace in title...
// IF artist matches "Elephant Gym"
// THEN Run regex search-replace ELSE skip this rule
Top-level hooks can also be an array of hooks. This makes creating multiple scenarios for top-level `when`-gated hooks easier. All hooks in the array will be run (assuming their `when`'s pass, if they exist) and their **input will be the Play object output of the previous hook in the array.**
<details>
<summary>Example</summary>
```json5
{
"sourceDefaults": {
"playTransform": {
"preCompare": [
// first lifecycle hook of preCompare to run
{
"title": [
{
"search": "something",
"replace": "else unique"
}
]
},
// second lifecycle hook of preCompare to run
{
"title": [
{
"search": "else unique",
"replace": "very demure"
}
]
},
]
}
}
}
```
</details>
### Logging
MS can log the output of hook transformations if/when they occur. In the `playTransform` object of a Source/Client config use `log`:
* `"log": true` => Output original play + final transformed output of last hook in the array
* `"log": "all"` => Output original play + final transformed output of **each** hook in the array
```json5
{
"name": "myThing",
"data": {/*...*/},
"options": {
"playTransform": {
"preCompare": {/*...*/},
"log": true
}
}
}
```
## Examples
### Remove phrase from Title in all new Plays
Removes the phrase `(Album Version)` from the Title of a Play
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"title": [
"(Album Version)"
]
}
}
}
}
```
</details>
### Remove all parenthesized content from the end of a title
<details>
<summary>Example</summary>
```json5 title="lastfm.json"
[
{
"name": "myLastFm",
// ...
"options": {
"playTransform": {
"compare": {
"candidate": {
"title": [
"/(\(.+\))\s*$/"
]
},
"existing": {
"title": [
"/(\(.+\))\s*$/"
]
},
},
}
}
}
]
```
</details>
### Rename misspelled artist in all new Plays
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"artists": [
{
"search": "Boz Skaggs",
"replace": "Boz Scaggs"
}
]
}
}
}
}
```
</details>
### Remove "Various Artists" albums in all new Plays
<details>
<summary>Example</summary>
```json5 title="config.json"
{
"sourceDefaults": {
"playTransform": {
"preCompare": {
"album": [
{
"search": "Various Artists",
"replace": ""
}
]
}
}
}
}
```
</details>
### Extract primary Artist from delimited, multi-Artist string
<details>
When the Artist string is actually a multi-artist, delimited string, this search-and-replace will replace the string with just the first artist found.
Ex
```
My Artist One / My Artist Two / Another Guy
My Artist One
```
Artists are delimited with a spaced forward slash (`/`) in the regex below. Replace the contents of the `delim` capture group with the delimiter for your use case. Some more common scenarios:
* `(?<delim>\\/)` No spaces between slash IE `My Artist One/My Artist Two/Another Guy`
* `(?<delim>\\s*\\\\\s*)` Backslash instead of forward slash IE `My Artist One \ My Artist Two \ Another Guy`
* `(?<delim>,)` Comma IE `My Artist One, My Artist Two, Another Guy`
Multi-scrobbler is written entirely in [Typescript](https://www.typescriptlang.org/). It consists of a backend and frontend. The backend handles all Source/Client logic, mounts web server endpoints that listen for Auth callbacks and Source ingress using [expressjs](https://expressjs.com/), and serves the frontend. The frontend is a standalone [Vitejs](https://vitejs.dev/) app that communicates via API to the backend in order to render the dashboard.
## Project Setup
Development requires [Node v18.19.1](https://nodejs.org/en) or higher is installed on your system.
:::tip
When running locally (not with a devcontainer) you can use [nvm](https://github.com/nvm-sh/nvm) to manage the installed node version.
:::
Clone this repository somewhere and then install from the working directory
nvm use # optional, sets correct node version when running without devcontainer
npm install
npm run start
```
### VSCode
This repository contains [workspace settings](https://github.com/FoxxMD/multi-scrobbler/blob/master/.devcontainer) for development with VSCode. These include:
* Run/Debug [Launch configurations](https://code.visualstudio.com/Docs/editor/debugging#_launch-configurations) for the application and tests
* [Devcontainer](https://code.visualstudio.com/docs/devcontainers/containers) for development with all dependencies already installed
* Useful extensions for linting and running tests
To use the Devcontainer simple open the repository in VSCode and "Use Devcontainer" when the notification is presented. `npm install` will be run when a new container is created.
## Common Development
:::info
In this document, when referring to aspects of Sources and Clients that are shared between both, the Source/Client will be referred to as a **Component.**
:::
A Component is composed of two parts:
* Typescript interfaces describing structure of configuration for that Component
* A concrete class inheriting from a common "startup" abstract class that enforces how the Component is built and operates
In both parts Source/Clients share some common properties/behavior before diverging in how they operate.
### Config
The configuration for a Component should always have this minimum shape, enforced respectively by the interfaces [CommonSourceConfig](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/common/infrastructure/config/source/index.ts#L105) and [CommonClientConfig](https://github.com/FoxxMD/multi-scrobbler/blob/ce1c70a4e1e87fb5bea7cca960eaafbd15881a1f/src/backend/common/infrastructure/config/client/index.ts#L68):
```ts
interfaceMyConfig{
name: string
data?: object
options?: object
}
```
*`data` contains data that is required for a Component to operate such as credentials, callback urls, api keys, endpoints, etc...
*`options` are **optional** settings that can be used to fine-tune the usage of the Component but are not required or do not majorly affect behavior. EX additional logging toggles
### Concrete Class
Components inherit from an abstract base class, [`AbstractComponent`](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/common/AbstractComponent.ts), that defines different "stages" of how a Component is built and initialized when MS first starts as well as when restarting the Component in the event it stops due to an error/network failure/etc...
#### Stages
Stages below are invoked in the order listed. All stages are asynchronous to allow fetching network requests or reading files.
The stage function (described in each stage below) should return a value or throw:
* return `null` if the stage is not required
* return `true` if the stage succeeded
* return a `string` if the stage succeeded and you wish to append a result to the log output for this stage
* throw an `Exception` if the stage failed for any reason and the Component should not continue to run/start up
##### Stage: Build Data
This stage should be used to validate user configuration, parse any additional data from async sources (file, network), and finalize the shape of any configuration/data needed for the Component to operate.
:::info
Implement [`doBuildInitData`](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/common/AbstractComponent.ts#L71) in your child class to invoke this stage.
::::
<details>
<summary>Examples</summary>
* Parse a full URL like `http://SOME_IP:7000/subfolder/api` from user config containing a base url like `data.baseUrl: 'SOME_IP'` and then store this in the class config
* Validate that config `data` contains required properties `user``password``salt`
* Read stored credentials from `${this.configDir}/currentCreds-MySource-${name}.json`;
</details>
##### Stage: Check Connection
This stage is used to validate that MS can communicate with the service the Component is interacting with. This stage is invoked on MS startup as well as any time the Component tries to restart after a failure.
If the Component depends on **ingress** (like Jellyfin/Plex webhook) this stage is not necessary.
:::info
Implement [`doCheckConnection`](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/common/AbstractComponent.ts#L103) in your child class to invoke this stage.
::::
<details>
<summary>Examples</summary>
* Make a [`request`](https://nodejs.org/docs/latest-v18.x/api/http.html#httprequesturl-options-callback) to the service's server to ensure it is accessible
* Open a websocket connection and check for a ping-pong
</details>
##### Stage: Test Auth
MS determines if Auth is required for a Component based on two class properties. You should set these properties during `constructor` initialization for your Component class:
*`requiresAuth` - (default `false`) Set to `true` if MS should check/test Auth for this Component
*`requiresAuthInteraction` - (default `false`) Set to `true` if user interaction is required to complete auth IE user needs to visit a callback URL
If the Component requires authentication in order to communicate with a service then any required data should be built in this stage and a request made to the service to ensure the authentication data is valid.
This stage should return:
*`true` if auth succeeded
*`false` if auth failed without unexpected errors
* IE the authentication data is not valid and requires user interaction to resolve the failure
* throw an exception if network failure or unexpected error occurred
You _should_ attempt to re-authenticate, if possible. Only throw an exception or return `false` if there is no way to recover from an authentication failure.
:::info
Implement [`doAuthentication`](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/backend/common/AbstractComponent.ts#L111) in your child class to invoke this stage.
::::
<details>
<summary>Examples</summary>
* Generate a Bearer Token for Basic Auth from user/password given in config and store in class properties
* Make a request to a known endpoint with Authorization token from read credentials file to see if succeeds or returns 403
* Catch a 403 and attempt to reauthenticate at an auth endpoint with user/password given in config
</details>
### Play Object
The **PlayObject** is the standard data structure MS uses to store listen (track) information and data required for scrobbling. It consists of:
* Track Data -- a standard format for storing track, artists, album, track duration, the date the track was played at, etc...
* Listen Metadata -- Optional but useful data related to the specific play or specifics about the Source/Client context for this play such as
* Platform specific ID, web URL to track, device/user ID that played this track, etc...
Both Sources and Clients use the **PlayObject** interface. When a Component receives track info from its corresponding service it must transform this data into a PlayObject before it can be interacted with.
For more refer to the TS documentation for `PlayObject` or [`AmbPlayObject`](https://github.com/FoxxMD/multi-scrobbler/blob/master/src/core/Atomic.ts#L141) in your project
## Creating Clients and Sources
* [Source Development and Tutorial](dev-source.md)
* [Client Development and Tutorial](dev-client.md)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
FoxxMD
