Compare commits

...
79 Commits
Author SHA1 Message Date
FoxxMD 7b0f5801a1 Merge branch 'develop' 2023-03-20 11:06:38 -04:00
FoxxMD 4dbea91a5e docs: some words 2023-03-15 12:28:44 -04:00
FoxxMD 548356aaa4 feat: Implement Kodi as a Source 2023-03-15 12:26:35 -04:00
FoxxMD 8b3f4c636c Merge branch 'develop' 2023-03-15 09:13:20 -04:00
FoxxMD 1b0212f3bb feat: Improve polling retry strategy
* Increase default to 5 for all sources
* Use true exponential backoff for determining delay between polling
2023-03-15 09:06:26 -04:00
FoxxMD 14e9ec0b45 refactor: Remove superfluous 'Client' from scrobbler label
Already know what it is since it is prefixed by [Scrobblers] label
2023-03-15 08:30:18 -04:00
FoxxMD dab113ab76 docs: Missing jriver TOC entry 2023-03-13 14:01:40 -04:00
FoxxMD ad729769ec Implement JRiver Source #26 2023-03-13 13:59:24 -04:00
FoxxMD bb4ae549a3 Remove superfluous spotify label 2023-03-13 10:13:47 -04:00
FoxxMD 0786642a38 Increase default poll retries for Spotify
This seems to be a pain point so increasing ought to help.
2023-03-13 10:09:22 -04:00
FoxxMD 77d5f5884e Use error with cause for spotify api errors 2023-03-13 10:00:23 -04:00
FoxxMD 7a0a23b7fd Merge branch 'develop' 2023-03-10 15:12:53 -05:00
FoxxMD 1207a5eec1 docs: Update wording 2023-03-10 14:58:27 -05:00
FoxxMD 48a0ecad41 feat: Implement Listenbrainz as a client #16 2023-03-10 14:38:07 -05:00
FoxxMD f792feeb6c fix: re-add accidentally delete lastfm docs 2023-03-10 14:28:50 -05:00
FoxxMD d6b682a23a fix: Do not prematurely return no existing scrobble when no recent scrobbles exist
don't want to assume we won't find an existing just because there are no scrobbles from the client itself -- we may have found it from recent recorded scrobbles sent
2023-03-10 14:26:42 -05:00
FoxxMD d77d547c15 fix: fix always adding plays to newly discovered 2023-03-10 14:25:05 -05:00
FoxxMD 0dc828fa85 fix: time frame should be valid if there were no recent scrobbles retrieved 2023-03-10 14:08:48 -05:00
FoxxMD bd00a5f414 feat: Improve error and json logging
* Properly handle ErrorWithCause stacks
* Output json when log meta exists
2023-03-10 12:41:07 -05:00
FoxxMD f587ad4bbc feat: Use fixed length array to store scrobbled plays
May reduce memory usage as we don't store all scrobbles ever
2023-03-10 11:58:57 -05:00
FoxxMD b8f2b1e3f8 feat: Increase number of recent scrobbles to fetch 2023-03-10 11:38:24 -05:00
FoxxMD bf3b81f1cd feat(listenbrainz): Add timeout for listens response 2023-03-10 11:34:41 -05:00
FoxxMD 9375a6fce4 feat: Improve listenbrainz scrobbler source of truth behavior
Use updated listenbrainz api methods and scrobbled play method
2023-03-10 11:24:15 -05:00
FoxxMD ca148f490e refactor: Make scrobbled data transform responsibility of concrete scrobbler class 2023-03-10 11:23:15 -05:00
FoxxMD 9ab0468913 feat: Improve listenbrainz api
* The response track data is different than submission data so make sure we are accurately parsing that data to PlayObjects
* Add separate method for getting listens as recently played (play objects) and refactor getUserListens to return raw payload
2023-03-10 11:22:21 -05:00
FoxxMD 99a73a80db feat: Implement Listenbrainz source 2023-03-10 09:54:20 -05:00
FoxxMD 80ba6e6c79 remove commented out code 2023-03-09 14:33:16 -05:00
FoxxMD 7a4f0f0d3f fix: Fix mopidy response handling when no track is being played 2023-03-09 13:12:31 -05:00
FoxxMD a9a6c6ae15 feat: Improve scrobble client debug logging during scrobble loop
Use client logger for cleaner labels
2023-03-09 13:09:26 -05:00
FoxxMD c5190fc2a6 feat: Improve logging
* use child loggers instead of creating new loggers for each part of the app
  * cleans up and automates nested labels in log output
  * fixes "event emitters may be leaky" node warning
* add ability to specify log levels per output type (console, ui, file) using aioconfig
* improve UI logging experience
  * use more color-blind friendly colors (borrowed from context-mod)
  * add verbose log level output
  * increase log buffer and store all logs, then filter at render time, instead of discarding based on min log level so that we always have logs to display
  * fix level selection in ui to work for all levels
2023-03-09 13:02:46 -05:00
FoxxMD 93b543e1a4 Fix mopidy json schema url 2023-03-08 15:21:10 -05:00
FoxxMD 4d77ca9c11 Add missing mopidy example 2023-03-08 15:18:24 -05:00
FoxxMD 03cf3d1e3c feat: Implement Mopidy source #24 2023-03-08 15:16:37 -05:00
FoxxMD d2134f3467 fix: fix reversing referenced array 2023-03-08 09:57:34 -05:00
FoxxMD 97fe411ac0 feat: improved last activity date tracking 2023-03-07 14:22:27 -05:00
FoxxMD ff953b4535 fix: oops left in debug statement 2023-03-07 12:31:27 -05:00
FoxxMD b68fb9a9f9 fix(youtube): Fix bad index when checking for repeat tracks 2023-03-07 12:20:02 -05:00
FoxxMD ad1895d6e1 fix: Add fallback comparison when sorting plays by playDate 2023-03-07 12:19:29 -05:00
FoxxMD 51b4ea3b16 Refactor: Consolidate formatPlayObj options 2023-03-07 11:51:36 -05:00
FoxxMD 3d86a38a2e Refactor: Create a Source of Truth for track discover by sources and consolidate/simplify scrobble behavior
* Add a 'recentDiscoveredPlays' class property for abstract source
  * recentDiscoveredPlays is platform aware and configured based on source 'multiPlatform' class property
* Consolidate scrobble behavior into abstract class
  * Check for existing discovered track before scrobbling
  * Add to discovered if not existing and increment discovered tracks counter
* Decouple sources from clients by using event emitter when a newly discovered track should be scrobbled
* Use discovered tracks for UI display instead of api data
2023-03-07 11:27:25 -05:00
FoxxMD 605aad1df3 Refactor: Consolidate scrobble client existing play check into abstract client
* Move main logic into class methods that can be overridden
* Move title, artist, and date checks into class methods that can be overridden
2023-03-07 10:36:05 -05:00
FoxxMD 3d2e3534ff Refactor: Move play date comparison into util 2023-03-07 10:34:20 -05:00
FoxxMD 2a65ffe6c4 Merge branch 'statefulSOT' into experimental 2023-03-06 13:20:37 -05:00
FoxxMD e3e3e03249 Merge branch 'spotifyConnect' into experimental
# Conflicts:
#	src/sources/AbstractSource.ts
#	src/sources/SpotifySource.ts
#	src/sources/SubsonicSource.ts
2023-03-06 13:20:30 -05:00
FoxxMD 6bd8642c91 Initial pass at IOC 2023-03-06 13:13:25 -05:00
FoxxMD 40ae842f82 Add fixed-size-list
To be used for storing a fixed length array of scrobbled plays for clients and a discovered tracks for sources
2023-03-06 11:12:11 -05:00
FoxxMD a2ab1aa49e feat: Add polling options to config schema and make polling more aggressive
Since spotify now uses currently playing with memory we need to poll more aggressively to make sure we catch tracks as they are played. Additionally need to decrease max polling interval so as not to miss activity.
2023-03-03 12:32:16 -05:00
FoxxMD 6d3cbf32df feat(spotify): Refactor spotify Source as Memory Source #51
* Use recently-played to scrobble backlogged tracks
* Use currently-playing or current-state to implement spotify as a memory source in order to track plays made from spotify connect devices
2023-03-02 15:29:31 -05:00
FoxxMD 7a8138773c feat(spotify): Implement currently playing retrieval 2023-03-02 13:38:56 -05:00
FoxxMD 5ae79deca4 feat: Implement MPRIS Source 2023-03-02 12:57:29 -05:00
FoxxMD 285f593a0f Add dbus-next dependency 2023-03-02 10:29:46 -05:00
FoxxMD e0ae621b62 feat: Implement more context awareness for memory sources to handle multi-platform/device plays
* Track candidate plays separately based on device/platform/user metadata
* Perform additional check for playback progress if metadata exists

Fixes #67
2023-02-28 12:34:09 -05:00
FoxxMD 259d4dabc8 feat: Parse device id metadata from sources to be used for multi-player awareness #67 2023-02-28 11:12:15 -05:00
FoxxMD f7b03d9622 refactor: Rename sourceId to trackId to be clearer in what it actually is 2023-02-28 10:14:06 -05:00
FoxxMD bd188a8ca7 refactor: Remove unused trackLength property 2023-02-28 09:58:56 -05:00
FoxxMD 28910a9332 Merge branch 'develop' 2023-02-27 11:13:28 -05:00
FoxxMD ee38252e1a docs: Add webhook and healthcheck endpoint documentation 2023-02-27 10:58:39 -05:00
FoxxMD 6f3595db31 feat: Add initialization and auth tests for notifiers 2023-02-27 10:58:31 -05:00
FoxxMD c5c0cd9cd2 refactor: Improve title and messages for webhook notifications 2023-02-27 10:09:25 -05:00
FoxxMD f5da32d233 fix: YT Music error throwing 2023-02-27 09:35:44 -05:00
FoxxMD 315869dafe fix: Update auth check when starting polling
Use general requiresAuth instead of only requiresAuthInteraction
2023-02-27 09:35:29 -05:00
FoxxMD 330802e29b fix: Default import for gotify library 2023-02-27 09:34:52 -05:00
FoxxMD 3d5e988ef0 Suppress error 2023-02-24 15:31:43 -05:00
FoxxMD 9a2b4b3d1e fix: Maybe fix for missing lastfm credentials configdir base 2023-02-24 15:27:16 -05:00
FoxxMD 4a1ebcdbc6 fix: Fix webhook missing initializer value 2023-02-24 13:57:08 -05:00
FoxxMD bc66fdb2b6 fix: Stop api calls if authentication failed 2023-02-24 13:56:52 -05:00
FoxxMD 6c2b0557c6 feat: Implement health endpoint #66
* HTTP Status as primary indicator with messages in json response
* No parameters in request aggregates all client/source statuses. Otherwise can use 'type' or 'name' parameters to restrict client/sources to aggregate
2023-02-24 13:51:59 -05:00
FoxxMD c24dc41811 feat: Implement webhook-based status notifications #66
* Implemented notifications for Gotify and Ntfy
* Notifications pushed on: polling started, polling retry, polling stopped on error, scrobble client scrobble failure
2023-02-24 13:19:03 -05:00
FoxxMD 0a41d32084 fix: Fix config deconstruct for polling data 2023-02-24 11:13:58 -05:00
FoxxMD b7aa087b74 feat: Add debug logging of jellyfin webhook payload with docs 2023-02-24 11:04:47 -05:00
FoxxMD 1f511d964d feat: Improve jellyfin logging
* Refactor initial connection info to be more verbose and only log once
* Refactor valid event check logging order of operations and make logging more verbose
2023-02-24 10:48:40 -05:00
FoxxMD 14eabecb7d Update google headers image 2023-02-23 12:57:11 -05:00
FoxxMD 8e31aa431d docs: Add youtube music docs and example 2023-02-23 12:54:48 -05:00
FoxxMD e47a636c2d feat: Add friendly auth error and hints for YT authentication test 2023-02-23 12:27:37 -05:00
FoxxMD bdd14b16de feat: Implement YT recent played UI 2023-02-23 12:16:44 -05:00
FoxxMD 8cc5aacb9a feat: Implement working YT Music source #34 2023-02-23 12:07:06 -05:00
FoxxMD d8e861dd54 feat: Add YTMusic api implementation and basic source 2023-02-22 16:35:09 -05:00
FoxxMD e369f9eb21 Merge branch 'develop' 2023-02-22 12:55:23 -05:00
FoxxMD ce6246e479 docs: Add FAQ to bug report template 2023-02-22 12:55:14 -05:00
83 changed files with 9272 additions and 1051 deletions
+3 -1
View File
@@ -7,6 +7,8 @@ assignees: ''
---
[Please check the FAQ](https://github.com/FoxxMD/multi-scrobbler/blob/master/docs/FAQ.md) before submitting a bug report.
**Describe the bug**
A clear and concise description of what the bug is.
@@ -21,7 +23,7 @@ Steps to reproduce the behavior:
A clear and concise description of what you expected to happen.
**Logs**
If possible reproduce the issue with [debug logging ON](/docs/FAQ.md#turn-on-debug-logging)
If possible reproduce the issue with [debug logging ON](https://github.com/FoxxMD/multi-scrobbler/blob/master/docs/FAQ.md#turn-on-debug-logging)
```
Copy and paste as much log data as possible related to this issue here.
+16 -8
View File
@@ -4,34 +4,42 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker Pulls](https://img.shields.io/docker/pulls/foxxmd/multi-scrobbler)](https://hub.docker.com/r/foxxmd/multi-scrobbler)
A javascript app to scrobble plays from multiple sources to [Maloja](https://github.com/krateng/maloja), [Last.fm](https://www.last.fm), and other clients (eventually!)
A javascript app to scrobble music you listened to, to [Maloja](https://github.com/krateng/maloja), [Last.fm](https://www.last.fm), and [ListenBrainz](https://listenbrainz.org)
* Supports scrobbling for many sources
* Supports scrobbling from many **Sources**
* [Spotify](/docs/configuration.md#spotify)
* [Plex](/docs/configuration.md#plex) or [Tautulli](/docs/configuration.md#tautulli)
* [Subsonic-compatible APIs](/docs/configuration.md#subsonic) (like [Airsonic](https://airsonic.github.io/))
* [Jellyfin](/docs/configuration.md#jellyfin)
* [Youtube Music](/docs/configuration.md#youtube-music)
* [Last.fm](/docs/configuration.md#lastfm-source)
* [ListenBrainz](/docs/configuration.md#listenbrainz--source-)
* [Deezer](/docs/configuration.md#deezer)
* Supports scrobbling to many clients
* [MPRIS (Linux Desktop)](/docs/configuration.md#mpris)
* [Mopidy](/docs/configuration.md#mopidy)
* [JRiver](/docs/configuration.md#jriver)
* [Kodi](/docs/configuration.md#kodi)
* Supports scrobbling to many **Clients**
* [Maloja](/docs/configuration.md#maloja)
* [Last.fm](/docs/configuration.md#lastfm)
* [ListenBrainz](/docs/configuration.md#listenbrainz)
* Monitor status of Sources and Clients using [webhooks (Gotify or Ntfy)](/docs/configuration.md#webhook-configurations) or [healthcheck endpoint](/docs/configuration.md#health-endpoint)
* Supports configuring for single or multiple users (scrobbling for your friends and family!)
* Web server interface for stats, basic control, and detailed logs
* Smart handling of credentials (persistent, authorization through app)
* Easy configuration through ENVs or JSON
* Built for Docker and unattended use!
* Docker images for x86/ARM
**Why should I use this over a browser extension and/or mobile app scrobbler?**
* **Platform independent** -- Because multi-scrobbler communicates directly with service APIs it will scrobble everything you play regardless of where you play it. No more need for apps on every platform you use!
* **Open-source** -- Get peace of mind knowing exactly how your personal data is being handled.
* **Consolidate play sources** -- Scrobble from many sources to one client with ease and without duplicating tracks.
* **Track your activity regardless of where you listen** -- Scrobble from many Sources to one Client with ease and without duplicating tracks.
* **Manage scrobbling for others** -- Scrobble for your friends and family without any setup on their part. Easily silo sources to specific clients to keep plays separate.
**But I already scrobble my music to Last.fm, is multi-scrobbler for me?**
**But I already scrobble my music to Last.fm/ListenBrainz, is multi-scrobbler for me?**
Yes! You can use [Last.fm as a **Source**](/docs/configuration.md#lastfm--source-) to mirror scrobbles from your Last.fm profile to Maloja. That way you can keep your current scrobble setup as-is but still get the benefit of capturing your data to a self-hosted location.
Yes! You can use [Last.fm as a **Source**](/docs/configuration.md#lastfm--source-) or [Listenbrainz as a **Source**](/docs/configuration.md#listenbrainz--source-) to forward scrobbles from your profile to any other Client! That way you can keep your current scrobble setup as-is but still get the benefit of capturing your data to a self-hosted location.
<img src="/assets/status-ui.jpg" width="800">
@@ -41,7 +49,7 @@ You set up configurations for one or more **Sources** and one or more **Clients*
### Source
A **Source** is a data source that contains information about tracks you are playing like a music player or platform. Examples are **Spotify, Jellyfin, Plex, Airsonic**, etc...
A **Source** is a data source that contains information about tracks you are playing like a music player or platform. Examples are **Spotify, Jellyfin, Plex, Youtube Music, Airsonic**, etc...
Source configurations consist of:
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

+29 -3
View File
@@ -14,9 +14,9 @@
"clients": ["myConfig"],
"name": "mySpotifySource",
"data": {
"clientId": "a89cba1569901a0671d5a9875fed4be1",
"clientSecret": "ec42e09d5ae0ee0f0816ca151008412a",
"redirectUri": "http://localhost:9078/callback"
"clientId": "a89cba1569901a0671d5a9875fed4be1",
"clientSecret": "ec42e09d5ae0ee0f0816ca151008412a",
"redirectUri": "http://localhost:9078/callback"
}
}
],
@@ -29,5 +29,31 @@
"apiKey": "myMalojaKey"
}
}
],
"webhooks": [
{
"name": "FirstGotifyServer",
"type": "gotify",
"url": "http://localhost:8070",
"token": "MyGotifyToken",
"priorities": {
"info": 5,
"warn": 7,
"error": 10
}
},
{
"type": "ntfy",
"name": "MyNtfyFriendlyNameForLogs",
"url": "http://localhost:9991",
"topic": "MyMultiScrobblerTopic",
"username": "Optional",
"password": "Optional",
"priorities": {
"info": 3,
"warn": 4,
"error": 5
}
}
]
}
+4 -1
View File
@@ -4,7 +4,10 @@
"clients": [],
"data": {
"users": ["FoxxMD"],
"servers": ["myServer","anotherServer"]
"servers": ["myServer","anotherServer"],
"options": {
"logPayload": false
}
}
}
]
+10
View File
@@ -0,0 +1,10 @@
[
{
"name": "MyJriver",
"data": {
"url": "0.0.0.0",
"username": "auser",
"password": "apassword"
}
}
]
+10
View File
@@ -0,0 +1,10 @@
[
{
"name": "MyKodi",
"data": {
"url": "http://localhost:8080/jsonrpc",
"username": "myUser",
"password": "myPassword"
}
}
]
+10
View File
@@ -0,0 +1,10 @@
[
{
"name": "brainz",
"configureAs": "client"
"data": {
"token": "029b081ba-9156-4pe7-88e5-3be671f5ea2b",
"username": "FoxxMD"
}
}
]
+11
View File
@@ -0,0 +1,11 @@
[
{
"name": "MyMopidy",
"data": {
"url": "localhost",
"uriBlacklist": [],
"uriWhitelist": [],
"albumBlacklist": []
}
}
]
+9
View File
@@ -0,0 +1,9 @@
[
{
"name": "ubuntu",
"data": {
"whitelist": ["vlc", "mpd"],
"blacklist": ["spotify"]
}
}
]
+10
View File
@@ -0,0 +1,10 @@
[
{
"name": "MyYTMusic",
"clients": [],
"data": {
"cookie": "VISITOR_INFO1_LIVE=jMDXz2_L8rY; __Secure-3PAPISID=3AxsXpSXGqOInSDn1jEKn; DEVICE_INFO=ChxOekU0TmTBpjek5EWZ0G; YSC=7gZdl3Zdl3; SID=TwhNsaZRXYTAtXxzGyu6rZdpg2HvGROeW8J4Ym_FhkhoZMUYEQ.; __Secure-1PSID=TwhNOsaZRXYTyRBe4rxAtXRIKsIEtk_Qot2VLBNfHQrQ.; __Secure-3PSID=ZRXYTAtXRIKsIEtk_Qot2yRBerZdpg2HvvZRXYTAtXRIKsIEtk_Qot2yRBerkuZICFQ.; HSID=A1UMmELW79; SSID=AKhomOs; APISID=IlHHmuzkPdQzZZDhHn3; SAPISID=3AxsXpy0u75Qb/n1jEKn; __Secure-1PAPISID=3AxsXpQb/AkSDn1jEKn; LOGIN_INFO=AFmP6vFpyVCZZAIgDwbkhWMBBhluaIWAPP:QUQ314UW5NWMjNmd2ZUJnYnJsakdIMjZoaE5zVVMjNmd2ZZUiHRlb3ZlV3ZIcUVyRVIMjNmdjNmd2ZZUivYlNqX2ZNZUiHdUNFNFdaYmJIW1NkJRX3hqdlU2YnFESkFuSS1uTldnZVRmLXNjWFc5OUJuR3dTd3JsZGZYa2EtZFQ2a0k2Ry1KQQ==; PREF=volume=26; SIDCC=AFvI_94PxXwls-ndqpGfPgFX3FWj80y_94PxXwls-ndqfSh15sP; __Secure-1PSIDCC=AFvIBnUbRr96I96UCIp2U4T8HRVk2B0HfKzhzxwsiP; __Secure-3PSIDCC=AFvIB3bINuUN0ETDR9gO91wpwWIVmpGki3BxT3bINuUN0ETDR9gO91wCH",
"authUser": "0",
}
}
]
+36 -1
View File
@@ -1,6 +1,13 @@
* [Connection Issues](#connection-issues)
* [Plex/Tautulli/Jellyfin don't connect](#plextautullijellyfin-dont-connect)
* [Jellyfin has warnings about undefined or missing data](#jellyfin-has-warnings-about-undefined-or-missing-data)
* [Spotify/Deezer/LastFM won't authenticate](#spotifydeezerlastfm-wont-authenticate)
* [Configuration Issues](#configuration-issues)
* [Config could not be parsed](#config-could-not-be-parsed)
# Connection Issues
## Plex/Tautulli/Jellyfin don't work
## Plex/Tautulli/Jellyfin don't connect
These three [sources](/README.md#source) are **ingress-based** which means that multi-scrobbler waits for the Plex/Tautulli/Jellyfin server to contact multi-scrobbler, as opposed to multi-scrobbler contacting the server.
@@ -59,6 +66,34 @@ Check the command-line output of the application or docker logs.
Administration -> Dashboard -> Advanced -> Logs
## Jellyfin has warnings about undefined or missing data
Make sure you have
* [Configured the webhook plugin correctly](/docs/configuration.md#jellyfin)
* Checked the **Send All Properties(ignores template)** option in the webhook settings and **Saved**
multi-scrobbler is known to work on Jellyfin `10.8.9` with Webhook version `11.0.0.0`.
You can verify the payload sent from the webhook by modifying your jellyfin configuration to include `logPayload: true` which will output the raw payload to DEBUG level logging:
```json
[
{
"name": "MyJellyfin",
"clients": [],
"data": {
"users": ["FoxxMD"],
"options": {
"logPayload": true
}
}
}
]
```
If your issue persists and you open an Issue for it please include the raw payload logs in your report.
## Spotify/Deezer/LastFM won't authenticate
Ensure any **client id** or **secrets** are correct in your configuration.
+430 -18
View File
@@ -5,33 +5,25 @@
* [Specific File Configuration](#specific-file-configuration)
* [Source Configurations](#source-configurations)
* [Spotify](#spotify)
* [ENV-Based](#env-based)
* [File-Based](#file-based)
* [Plex](#plex)
* [ENV-Based](#env-based-1)
* [File-Based](#file-based-1)
* [Tautulli](#tautulli)
* [ENV-Based](#env-based-2)
* [File-Based](#file-based-2)
* [Subsonic](#subsonic)
* [ENV-Based](#env-based-3)
* [File-Based](#file-based-3)
* [Jellyfin](#jellyfin)
* [ENV-Based](#env-based-4)
* [File-Based](#file-based-4)
* [Last.fm (Source)](#lastfm--source-)
* [ENV-Based](#env-based-5)
* [File-Based](#file-based-5)
* [Listenbrainz (Source)](#listenbrainz--source-)
* [Deezer](#deezer)
* [ENV-Based](#env-based-6)
* [File-Based](#file-based-6)
* [Youtube Music](#youtube-music)
* [MPRIS (Linux Desktop)](#mpris)
* [Mopidy](#mopidy)
* [JRiver](#jriver)
* [Kodi](#kodi)
* [Client Configurations](#client-configurations)
* [Maloja](#maloja)
* [ENV-Based](#env-based-7)
* [File-Based](#file-based-7)
* [Last.fm](#lastfm)
* [ENV-Based](#env-based-8)
* [File-Based](#file-based-8)
* [Listenbrainz](#listenbrainz)
* [Monitoring](#monitoring)
* [Webhooks](#webhook-configurations)
* [Health Endpoint](#health-endpoint)
# Configuration Overview
@@ -280,6 +272,20 @@ No support for ENV based for Last.fm as a client (only source)
See [`lastfm.json.example`](/config/lastfm.json.example), change `configureAs` to `source`. Or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FLastfmSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Listenbrainz (Source)](https://listenbrainz.org)
You will need to run your own Listenbrainz server or have an account [on the official instance](https://listenbrainz.org/login/)
On your [profile page](https://listenbrainz.org/profile/) find your **User Token** to use in the configuration.
**NOTE:** You cannot use ENV variables shown in the [Listenbrainz Client config](#listenbrainz) -- multi-scrobbler assumes Listenbrainz ENVs are always used for the **client** configuration. You must use the file-based config from below to setup Listenbrainz as a Source.
### File-Based
See [`listenbrainz.json.example`](/config/listenbrainz.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FListenBrainzSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
**Change `configureAs` to `source`**
## [Deezer](https://deezer.com/)
Create a new application at [Deezer Developers](https://developers.deezer.com/myapps)
@@ -310,6 +316,291 @@ After starting multi-scrobbler with credentials in-place open the dashboard (`ht
See [`deezer.json.example`](/config/deezer.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FDeezerSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Youtube Music](https://music.youtube.com)
Credentials for YT Music are obtained from a browser request to https://music.youtube.com **once you are logged in.** [Specific requirements are here and summarized below:](https://github.com/nickp10/youtube-music-ts-api/blob/master/DOCUMENTATION.md#authenticate)
* Open a new tab
* Open the developer tools (Ctrl-Shift-I) and select the “Network” tab
* Go to https://music.youtube.com and ensure you are logged in
Then...
1. Find and select an authenticated POST request. The simplest way is to filter by /browse using the search bar of the developer tools. If you dont see the request, try scrolling down a bit or clicking on the library button in the top bar.
2. **Make sure **Headers** pane is selected and open
3. In the **Request Headers** section find and copy the **entire value** found after `Cookie:` and use this as the `cookie` value in your multi-scrobbler config
4. If present, in the **Request Headers** section find and copy the number found in `X-google-AuthUser` and use this as the value for `authUser` in your multi-scrobbler config
![Google Headers](/docs/google-header.jpg)
NOTES:
* YT Music authentication is "browser based" which means your credentials may expire after a (long?) period of time OR if you log out of https://music.youtube.com. In the event this happens just repeat the steps above to get new credentials.
* Communication to YT Music is **unofficial** and not supported or endorsed by Google. This means that **this integration may stop working at any time** if Google decides to change how YT Music works in the browser.
### File-Based
See [`ytmusic.json.example`](/config/ytmusic.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FYTMusicSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [MPRIS](https://specifications.freedesktop.org/mpris-spec/latest/)
MPRIS is a standard interface for communicating with Music Players on **linux operating systems.**
If you run Linux and have a notification tray that shows what media you are listening to, you likely have access to MPRIS.
![Notification Tray](/assets/mpris.jpg)
multi-scrobbler can listen to this interface and scrobble tracks played by **any media player** that communicates to the operating system with MPRIS.
**NOTE:** multi-scrobbler needs to be running as a [**Local Installation**](/docs/installation.md#local) in order to use MPRIS. This cannot be used from docker.
### ENV-Based
| Environmental Variable | Required? | Default | Description |
|------------------------|-----------|---------|----------------------------------------------------------------------------------|
| MPRIS_ENABLE | No | | Use MPRIS as a Source (useful when you don't need any other options) |
| MPRIS_BLACKLIST | No | | Comma-delimited list of player names not to scrobble from |
| MPRIS_WHITELIST | No | | Comma-delimited list of players names to ONLY scrobble from. Overrides blacklist |
### File-Based
See [`mpris.json.example`](/config/mpris.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FMPRISSourceConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Mopidy](https://mopidy.com/)
Mopidy is a headless music server that supports playing music from many [standard and non-standard sources such as Pandora, Bandcamp, and Tunein.](https://mopidy.com/ext/)
multi-scrobbler can scrobble tracks played from any Mopidy backend source, regardless of where you listen to them.
### File-Based
See [`mopidy.json.example`](/config/mopidy.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FMopidySourceConfig/%23%2Fdefinitions%2FMopidyData?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
Configuration Options:
##### `url`
The URL used to connect to the Mopidy server. You MUST have [Mopidy-HTTP extension](https://mopidy.com/ext/http) enabled.
If no `url` is provided a default is used which assumes Mopidy is installed on the same server as multi-scrobbler: `ws://localhost:6680/mopidy/ws/`
Make sure the hostname and port number match what is found in the Mopidy configuration file `mopidy.conf`:
```
...
[http]
hostname = localhost
port = 6680
...
```
The URL used to connect ultimately must be formed like this: `[protocol]://[hostname]:[port]/[path]`
If any part of this URL is missing multi-scrobbler will use a default value, for your convenience. This also means that if any part of your URL is **not** standard you must explicitly define it.
Part => Default Value
* Protocol => `ws://`
* Hostname => `localhost`
* Port => `6680`
* Path => `/mopidy/ws/`
<details>
<summary>URL Transform Examples</summary>
```json
{
"url": "mopidy.mydomain.com"
}
```
MS transforms this to: `ws://mopidy.mydomain.com:6680/mopidy/ws/`
```json
{
"url": "192.168.0.101:3456"
}
```
MS transforms this to: `ws://192.168.0.101:3456/mopidy/ws/`
```json
{
"url": "mopidy.mydomain.com:80/MOPWS"
}
```
MS transforms this to: `ws://mopidy.mydomain.com:80/MOPWS`
</details>
#### URI Blacklist/Whitelist
If you wish to disallow or only allow scrobbling from some sources played through Mopidy you can specify these using `uriBlacklist` or `uriWhitelist` in your config. multi-scrobbler will check the list to see if any string matches the START of the `uri` on a track. If whitelist is used then blacklist is ignored. All strings are case-insensitive.
EX:
```json
{
"uriBlacklist": ["soundcloud"]
}
```
Will prevent multi-scrobbler from scrobbling any Mopidy track that start with a `uri` like `soundcloud:song:MySong-1234`
#### Album Blacklist
For certain sources (Soundcloud) Mopidy does not have all track info (Album) and will instead use "Soundcloud" as the Album name. You can prevent multi-scrobbler from using this bad Album data by adding the fake Album name to this list. Multi-scrobbler will still scrobble the track, just without the bad data. All strings are case-insensitive.
EX:
```json
{
"albumBlacklist": ["SoundCloud", "Mixcloud"]
}
```
If a track would be scrobbled like `Album: Soundcloud, Track: My Cool Track, Artist: A Cool Artist`
then multi-scrobbler will instead scrobble `Track: My Cool Track, Artist: A Cool Artist`
## [JRiver](https://jriver.com/)
In order for multi-scrobbler to communicate with JRiver you must have [Web Server Interface](https://wiki.jriver.com/index.php/Web_Service_Interface#Documentation_of_Functions) enabled. This can can be in the JRiver GUI:
* Tools -> Options -> Media Network
* Check `Use Media Network to share this library...`
* If you have `Authentication` checked you will need to provide the **Username** and **Password** in the ENV/File configuration below.
#### URL
If you do not provide a URL then a default is used which assumes JRiver is installed on the same server as multi-scrobbler: `http://localhost:52199/MCWS/v1/`
* Make sure the port number matches what is found in `Advanced` section in the [Media Network](#jriver) options.
* If your installation is on the same machine but you cannot connect using `localhost` try `0.0.0.0` instead.
The URL used to connect ultimately must be formed like this: `[protocol]://[hostname]:[port]/[path]`
If any part of this URL is missing multi-scrobbler will use a default value, for your convenience. This also means that if any part of your URL is **not** standard you must explicitly define it.
Part => Default Value
* Protocol => `http://`
* Hostname => `localhost`
* Port => `52199`
* Path => `/MCWS/v1/`
<details>
<summary>URL Transform Examples</summary>
```json
{
"url": "jriver.mydomain.com"
}
```
MS transforms this to: `http://jriver.mydomain.com:52199/MCWS/v1/`
```json
{
"url": "192.168.0.101:3456"
}
```
MS transforms this to: `http://192.168.0.101:3456/MCWS/v1/`
```json
{
"url": "mydomain.com:80/jriverReverse/MCWS/v1/"
}
```
MS transforms this to: `http://mydomain.com:80/jriverReverse/MCWS/v1/`
</details>
### ENV-Based
| Environmental Variable | Required | Default | Description |
|------------------------|----------|---------------------------------|------------------------------------------------|
| JRIVER_URL | Yes | http://localhost:52199/MCWS/v1/ | The URL of the JRiver server |
| JRIVER_USERNAME | No | | If authentication is enabled, the username set |
| JRIVER_PASSWORD | No | | If authenticated is enabled, the password set |
### File-Based
See [`jriver.json.example`](/config/jriver.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FJRiverSourceConfig/%23%2Fdefinitions%2FJRiverData?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
## [Kodi](https://kodi.tv/)
In order for multi-scrobbler to communicate with Kodi you must have the [Web Interface](https://kodi.wiki/view/Web_interface) enabled. This can can be in the Kodi GUI:
* Settings -> Services -> Control
* Check `Allow remote control via HTTP`
* Ensure you have a **Username** and **Password** set, you will need to provide them in the ENV/File configuration below.
#### URL
If you do not provide a URL then a default is used which assumes Kodi is installed on the same server as multi-scrobbler: `http://localhost:8080/jsonrpc`
* Make sure the port number matches what is found in **Port** in the [Control](#kodi) section mentioned above.
* If your installation is on the same machine but you cannot connect using `localhost` try `0.0.0.0` instead.
The URL used to connect ultimately must be formed like this: `[protocol]://[hostname]:[port]/[path]`
If any part of this URL is missing multi-scrobbler will use a default value, for your convenience. This also means that if any part of your URL is **not** standard you must explicitly define it.
Part => Default Value
* Protocol => `http://`
* Hostname => `localhost`
* Port => `8080`
* Path => `/jsonrpc`
<details>
<summary>URL Transform Examples</summary>
```json
{
"url": "kodi.mydomain.com"
}
```
MS transforms this to: `http://kodi.mydomain.com:8080/jsonrpc`
```json
{
"url": "192.168.0.101:3456"
}
```
MS transforms this to: `http://192.168.0.101:3456/jsonprc`
```json
{
"url": "mydomain.com:80/kodiReverse/jsonrpc"
}
```
MS transforms this to: `http://mydomain.com:80/kodiReverse/jsonrpc`
</details>
### ENV-Based
| Environmental Variable | Required | Default | Description |
|------------------------|----------|-------------------------------|----------------------------|
| KODI_URL | Yes | http://localhost:8080/jsonrpc | The URL of the Kodi server |
| KODI_USERNAME | No | | The username set |
| KODI_PASSWORD | No | | The password set |
### File-Based
See [`kodi.json.example`](/config/kodi.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FKodiSourceConfig/%23%2Fdefinitions%2FKodiData?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fsource.json)
# Client Configurations
## [Maloja](https://github.com/krateng/maloja)
@@ -347,3 +638,124 @@ or replace `localhost:9078` with your own base URL
### File-Based
See [`lastfm.json.example`](/config/lastfm.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23/%23%2Fdefinitions%2FLastfmClientConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fclient.json)
## [Listenbrainz](https://listenbrainz.org)
You will need to run your own Listenbrainz server or have an account [on the official instance](https://listenbrainz.org/login/)
On your [profile page](https://listenbrainz.org/profile/) find your **User Token** to use in the configuration.
### ENV-Based
| Environmental Variable | Required? | Default | Description |
|------------------------|-----------|-------------------------------|---------------------------------|
| LZ_TOKEN | Yes | | User token from your LZ profile |
| LZ_USER | Yes | | Your LZ username |
| LZ_URL | No | https://api.listenbrainz.org/ | The base URL for the LZ server |
### File-Based
See [`listenbrainz.json.example`](/config/listenbrainz.json.example) or [explore the schema with an example and live editor/validator](https://json-schema.app/view/%23%2Fdefinitions%2FListenBrainzClientConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Fclient.json)
# Monitoring
multi-scrobbler supports some common webhooks and a healthcheck endpoint in order to monitor Sources and Clients for errors.
## Webhook Configurations
Webhooks will **push** a notification to your configured servers on these events:
* Source polling started
* Source polling retry
* Source polling stopped on error
* Scrobble client scrobble failure
Webhooks are configured in the main [config.json](#all-in-one-file-configuration) file under the `webhook` top-level property. Multiple webhooks may be configured for each webhook type. EX:
```json
{
"sources": [
...
],
"clients": [
...
],
"webhooks": [
{
"name": "FirstGotifyServer",
"type": "gotify",
"url": "http://192.168.0.100:8070",
"token": "abcd"
},
{
"name": "SecondGotifyServer",
"type": "gotify",
...
},
{
"name": "NtfyServerOne",
"type": "ntfy",
...
},
...
]
}
```
### [Gotify](https://gotify.net/)
Refer to the [config schema for GotifyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FGotifyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info` `warn` and `error` mappings.
EX
```json
{
"type": "gotify",
"name": "MyGotifyFriendlyNameForLogs",
"url": "http://192.168.0.100:8070",
"token": "AQZI58fA.rfSZbm",
"priorities": {
"info": 5,
"warn": 7,
"error": 10
}
}
```
### [Ntfy](https://ntfy.sh/)
Refer to the [config schema for NtfyConfig](https://json-schema.app/view/%23/%23%2Fdefinitions%2FNtfyConfig?url=https%3A%2F%2Fraw.githubusercontent.com%2FFoxxMD%2Fmulti-scrobbler%2Fdevelop%2Fsrc%2Fcommon%2Fschema%2Faio.json)
multi-scrobbler optionally supports setting message notification priority via `info` `warn` and `error` mappings.
EX
```json
{
"type": "ntfy",
"name": "MyNtfyFriendlyNameForLogs",
"url": "http://192.168.0.100:9991",
"topic": "RvOwKJ1XtIVMXGLR",
"username": "Optional",
"password": "Optional",
"priorities": {
"info": 3,
"warn": 4,
"error": 5
}
}
```
## Health Endpoint
An endpoint for monitoring the health of sources/clients is available at GET `http://YourMultiScrobblerDomain/health`
* Returns `200 OK` when **everything** is working or `500 Internal Server Error` if **anything** is not
* The plain url (`/health`) aggregates status of **all clients/sources** -- so any failing client/source will make status return 500
* Use query params `type` or `name` to restrict client/sources aggregated IE `/health?type=spotify` or `/health?name=MyMaloja`
* On 500 the response returns a JSON payload with `messages` array that describes any issues
* For any clients/sources that require authentication `/health` will return 500 if they are **not authenticated**
* For sources that poll (spotify, yt music, subsonic) `/health` will 500 if they are **not polling**
Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

+2
View File
@@ -31,6 +31,8 @@ SPOTIFY_CLIENT_ID=yourId SPOTIFY_CLIENT_SECRET=yourSecret MALOJA_URL="http://dom
## [Docker](https://hub.docker.com/r/foxxmd/multi-scrobbler)
Cross-platform images are built for x86 (Intel/AMD) and ARM (IE Raspberry Pi)
```
foxxmd/multi-scrobbler:latest
```
+26
View File
@@ -6,6 +6,7 @@ Scenario:
* Each person has their own Maloja server
* Each person has their own Spotify account
* You have your own Airsonic (subsonic) server you to scrobble from
* You have your own Youtube Music account you want to scrobble from
* Mary has her own Last.fm account she also wants to scrobble to
* Fred has his own Spotify application and provides you with just his access and refresh token because he doesn't trust you (wtf Fred)
* Fred has a Plex server and wants to scrobble everything he plays
@@ -101,6 +102,15 @@ Using just one config file located at `CONFIG_DIR/config.json`:
"url": "https://airsonic.foxx.example"
}
},
{
"type": "ytmusic",
"name": "foxxYoutube",
"clients": ["foxxMaloja"],
"data": {
"cookie": "__Secure-3PAPISID=3AxsXpy0MKGu75Qb/AkISXGqOnSDn1jEKn; DEVICE_INFO=ChxOekU0Tmpjek5EWTBPRGd3TlRBMk16QXpNdz09EJbS8Z0GGJbS8Z0G; ...",
"authUser": 1
}
},
],
"clients": [
{
@@ -217,6 +227,22 @@ In `CONFIG_DIR/jellyfin.json`
]
```
In `CONFIG_DIR/ytmusic.json`
```json5
[
{
"type": "ytmusic",
"name": "foxxYoutube",
"clients": ["foxxMaloja"],
"data": {
"cookie": "__Secure-3PAPISID=3AxsXpy0MKGu75Qb/AkISXGqOnSDn1jEKn; DEVICE_INFO=ChxOekU0Tmpjek5EWTBPRGd3TlRBMk16QXpNdz09EJbS8Z0GGJbS8Z0G; ...",
"authUser": 1
}
}
]
```
In `CONFIG_DIR/maloja.json`:
```json5
+7 -1
View File
@@ -44,7 +44,13 @@ Expand the **Watched** dropdown and add the following code block to the **JSON D
"username": "{username}",
"server": "{server_name}",
"version": "{server_version}",
"library": "{library_name}"
"library": "{library_name}",
"player": "{player}",
"device": "{device}",
"platform": "{platform}",
"action": "{action}",
"machine_id": "{machine_id}",
"session_key": "{session_key}"
}
```
+1703 -97
View File
File diff suppressed because it is too large Load Diff
+16 -3
View File
@@ -36,31 +36,43 @@
"homepage": "https://github.com/FoxxMD/multi-scrobbler#readme",
"dependencies": {
"@awaitjs/express": "^0.6.3",
"@kenyip/backoff-strategies": "^1.0.4",
"ajv": "^7.2.4",
"body-parser": "^1.19.0",
"compare-versions": "^4.1.2",
"concat-stream": "^2.0.0",
"dayjs": "^1.10.4",
"dbus-next": "^0.10.2",
"ejs": "^3.1.6",
"es6-error": "^4.1.1",
"express": "^4.17.1",
"express-session": "^1.17.2",
"fixed-size-list": "^0.3.0",
"formidable": "^2.1",
"gotify": "^1.1.0",
"iti": "^0.6.0",
"json5": "^2.2.3",
"kodi-api": "^0.2.1",
"lastfm-node-client": "^2.2.0",
"mopidy": "^1.3.0",
"normalize-url": "^6.1.0",
"ntfy": "^1.0.5",
"p-event": "^4.2.0",
"passport": "^0.6.0",
"passport-deezer": "^0.2.0",
"pony-cause": "^1.1.1",
"safe-stable-stringify": "^1.1.1",
"socket.io": "4.1.3",
"socket.io": "^4.6.1",
"spotify-web-api-node": "^5.0.2",
"superagent": "^8.0.9",
"triple-beam": "^1.3.0",
"winston": "^3.3.3",
"winston": "github:FoxxMD/winston#fbab8de969ecee578981c77846156c7f43b5f01e",
"winston-daily-rotate-file": "^4.5.0",
"winston-duplex": "^0.1.1",
"winston-null": "^2.0.0",
"winston-transport": "^4.4.0"
"winston-transport": "^4.4.0",
"xml2js": "^0.4.23",
"youtube-music-ts-api": "^1.4.1"
},
"devDependencies": {
"@tsconfig/node18": "^1.0.1",
@@ -73,6 +85,7 @@
"@types/spotify-web-api-node": "^5.0.7",
"@types/superagent": "^4.1.16",
"@types/triple-beam": "^1.3.2",
"@types/xml2js": "^0.4.11",
"ts-essentials": "^9.1.2",
"ts-node": "^10.7.0",
"tsconfig-paths": "^3.13.0",
+5 -4
View File
@@ -1,6 +1,7 @@
import {capitalize, createLabelledLogger} from "../utils.js";
import {capitalize, mergeArr} from "../utils.js";
import {Logger} from "winston";
import {PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, PlayObject} from "../common/infrastructure/Atomic.js";
import winston from 'winston';
export default abstract class AbstractApiClient {
name: string;
@@ -19,12 +20,12 @@ export default abstract class AbstractApiClient {
this.type = type;
this.name = name;
const identifier = `API - ${capitalize(this.type)} - ${name}`;
this.logger = createLabelledLogger(identifier, identifier);
this.logger = winston.loggers.get('app').child({labels: [identifier]}, mergeArr);
this.config = config;
this.options = options;
}
static formatPlayObj = (obj: any): PlayObject => {
static formatPlayObj = (obj: any, options: FormatPlayObjectOptions): PlayObject => {
throw new Error('should be overridden');
}
}
+167
View File
@@ -0,0 +1,167 @@
import AbstractApiClient from "./AbstractApiClient.js";
import {JRiverData} from "../common/infrastructure/config/source/jriver.js";
import request, {Request, Response} from 'superagent';
import xml2js from 'xml2js';
import {ErrorWithCause} from "pony-cause";
const parser = new xml2js.Parser({'async': true});
export const PLAYER_STATE: Record<string, PLAYER_STATE> = {
STOPPED: '0',
PAUSED: '1',
PLAYING: '2'
}
export type PLAYER_STATE = '0' | '1' | '2';
interface JRiverResponseItem {
_: string
$: {
Name: string
}
}
interface JRiverResponse {
Response: {
'$': {
Status: string
},
Item: JRiverResponseItem[]
}
}
export interface JRiverTransformedResponse<T> {
status: string
data?: T
}
export interface Alive {
RuntimeGUID: string
LibraryVersion: string
ProgramName: string
ProgramVersion: string
FriendlyName: string
AccessKey: string
ProductVersion: string
Platform: string
}
// state 0 = nothing?
// 2 = playing
// 1 = paused
export interface Authenticate {
Token: string
ReadOnly: number
PreLicensed: boolean
}
export interface Info {
ZoneID: string
ZoneName: string
State: PLAYER_STATE
PositionMS: number
DurationMS: number
Artist: string
Album: string
Name: string
Status: string
FileKey: string
}
export interface Zones {
NumberZones: number
CurrentZoneID: string
CurrentZoneIndex: string
}
const jriverResponseTransform = <T>(val: JRiverResponse): JRiverTransformedResponse<T> => {
const status = val.Response.$.Status;
const items = val.Response.Item === undefined ? undefined : val.Response.Item.map(x => {
return [x.$.Name, x._];
});
return {
status,
data: items.reduce((acc, curr) => {
acc[curr[0]] = curr[1];
return acc;
}, {}) as T
};
}
export class JRiverApiClient extends AbstractApiClient {
declare config: JRiverData
url: string;
token?: string;
constructor(name: any, config: JRiverData, options = {}) {
super('JRiver', name, config, options);
const {
url = 'http://localhost:52199/MCWS/v1/'
} = config;
this.url = url;
}
callApi = async <T>(req: Request, retries = 0): Promise<Response & {body: T}> => {
const {
maxRequestRetries = 2,
retryMultiplier = 1.5
} = this.config;
if (this.token !== undefined) {
req.query({token: this.token});
}
try {
const resp = await req as Response;
if (resp.text !== '') {
const rawBody = await parser.parseStringPromise(resp.text);
resp.body = <T>jriverResponseTransform(rawBody);
}
return resp;
} catch (e) {
throw e;
}
}
testConnection = async () => {
try {
const resp = await this.callApi<Alive>(request.get(`${this.url}Alive`));
const {body: { data } = {}} = resp;
this.logger.verbose(`Found ${data.ProgramName} ${data.ProgramVersion} (${data.FriendlyName})`);
return true;
} catch (e) {
this.logger.error(new ErrorWithCause('Could not communicate with JRiver server. Verify your server URL is correct.', {cause: e}));
return false;
}
}
testAuth = async () => {
try {
let req = request.get(`${this.url}Authenticate`);
if (this.config.username !== undefined) {
req.auth(this.config.username, this.config.password);
}
const resp = await this.callApi<Authenticate>(req);
this.token = resp.body.data.Token;
return true;
} catch (e) {
let msg = 'Authentication failed.';
if(this.config.username === undefined || this.config.password === undefined) {
msg = 'Authentication failed. No username/password was provided in config! Did you mean to do this?';
}
this.logger.error(new ErrorWithCause(msg, {cause: e}));
return false;
}
}
getInfo = async (zoneId: string = '-1') => {
return await this.callApi<Info>(request.get(`${this.url}Playback/Info`).query({Zone: zoneId}));
}
getZones = async () => {
return await this.callApi<Zones>(request.get(`${this.url}Playback/Zones`));
}
}
+187
View File
@@ -0,0 +1,187 @@
import AbstractApiClient from "./AbstractApiClient.js";
import {ErrorWithCause} from "pony-cause";
import {KodiData} from "../common/infrastructure/config/source/kodi.js";
import { KodiClient } from 'kodi-api'
import normalizeUrl from "normalize-url";
import {URL} from "url";
import {RecentlyPlayedOptions} from "../sources/AbstractSource.js";
import {FormatPlayObjectOptions, PlayObject} from "../common/infrastructure/Atomic.js";
import dayjs from "dayjs";
interface KodiDuration {
hours: number
milliseconds: number
minutes: number
seconds: number
}
interface ActivePlayer {
playerid: number
playertype: string
type: string
}
interface PlayerInfo {
position: -1 | 0
time: KodiDuration
totaltime: KodiDuration
type: string
}
interface PlayerItem {
album: string
albumartist: string[]
artist: string[]
duration: number
id: number
label: string
title: string
type: string
}
export class KodiApiClient extends AbstractApiClient {
declare config: KodiData
url: URL;
token?: string;
client: KodiClient;
constructor(name: any, config: KodiData, options = {}) {
super('Kodi', name, config, options);
const {
url = 'http://localhost:8080/jsonrpc'
} = config;
this.url = KodiApiClient.parseConnectionUrl('http://localhost:8080');
const auth = new Buffer(`${config.username}:${config.password}`).toString('base64');
this.client = new KodiClient({
clientType: this.url.protocol.replace(':', '') as ('http' | 'https'),
//parameterMode: 'object',
clientOptions: {
host: this.url.hostname,
port: this.url.port,
path: this.url.pathname,
headers: {
Authorization: `Basic ${auth}`
},
}
});
}
static parseConnectionUrl(val: string) {
const normal = normalizeUrl(val, {removeTrailingSlash: true, normalizeProtocol: true})
const url = new URL(normal);
if (url.port === null || url.port === '') {
url.port = '8080';
}
if (url.pathname === '/') {
url.pathname = '/jsonrpc';
} else if (url.pathname === '/jsonrpc/') {
url.pathname = '/jsonrpc';
}
return url;
}
static formatPlayObj(obj: (PlayerItem & Partial<PlayerInfo> & { playerid?: number }), options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = true} = options;
const {
album: albumVal,
albumartist,
artist: artistVal,
duration,
id,
title,
time,
playerid,
} = obj;
let artists = artistVal === null || artistVal === undefined ? [] : artistVal;
let album = albumVal === null || albumVal === '' ? undefined : albumVal;
const trackProgressPosition = time !== undefined ? Math.round(dayjs.duration(time).asSeconds()) : undefined;
return {
data: {
track: title,
album: album,
artists,
duration,
playDate: dayjs()
},
meta: {
source: 'kodi',
trackId: id.toString(),
newFromSource,
trackProgressPosition,
deviceId: playerid !== undefined ? `Player${playerid}` : undefined,
}
}
}
testConnection = async () => {
return true;
}
testAuth = async () => {
try {
await this.client.connect();
// https://kodi.wiki/view/JSON-RPC_API/v12#JSONRPC.Version
const jsonInfo = await this.client.JSONRPC.Version();
// https://kodi.wiki/view/JSON-RPC_API/v12#Application.GetProperties
const applicationInfo = await this.client.Application.GetProperties(['version']);
this.logger.info(`Found Kodi v${applicationInfo.version.major}.${applicationInfo.version.minor} (JSONRPC v${jsonInfo.version.major}.${jsonInfo.version.minor})`);
return true;
} catch (e) {
let msg = 'Authentication failed.';
if(this.config.username === undefined || this.config.password === undefined) {
msg = 'Authentication failed. No username/password was provided in config! Did you mean to do this?';
}
this.logger.error(new ErrorWithCause(msg, {cause: e}));
return false;
}
}
getPlayerInfo = async (id: number): Promise<PlayerInfo> => {
// https://kodi.wiki/view/JSON-RPC_API/v12#Player.GetProperties
// @ts-ignore
const playerInfo = await this.client.Player.GetProperties(0, ["position","type","time","totaltime"])
return playerInfo;
}
getPlayerItem = async (id: number): Promise<{item: PlayerItem}> => {
// https://kodi.wiki/view/JSON-RPC_API/v12#Player.GetItem
// @ts-ignore
const itemInfo = await this.client.Player.GetItem(0, ["title","artist","album","albumartist","starttime","endtime","duration","streamdetails","uniqueid"]);
return itemInfo as {item: PlayerItem};
}
getActivePlayers = async (): Promise<ActivePlayer[]> => {
// https://kodi.wiki/view/JSON-RPC_API/v12#Player.GetActivePlayers
const players = await this.client.Player.GetActivePlayers();
return players as ActivePlayer[];
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}): Promise<PlayObject[]> => {
const active = await this.getActivePlayers();
if(active.length === 0) {
return [];
}
// get first active playing audio
const audioActive = active.find(x => x.type === 'audio');
if(audioActive === undefined) {
return [];
}
const playerInfo = await this.getPlayerInfo(audioActive.playerid);
const itemInfo = await this.getPlayerItem(audioActive.playerid);
const play = KodiApiClient.formatPlayObj({...itemInfo.item, ...playerInfo, ...audioActive});
return [play];
}
}
+3 -4
View File
@@ -2,7 +2,7 @@ import LastFm, {AuthGetSessionResponse, TrackObject, UserGetInfoResponse} from "
import AbstractApiClient from "./AbstractApiClient.js";
import dayjs from "dayjs";
import {readJson, sleep, writeFile} from "../utils.js";
import {PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, PlayObject} from "../common/infrastructure/Atomic.js";
import {LastfmData} from "../common/infrastructure/config/client/lastfm.js";
const badErrors = [
@@ -24,9 +24,8 @@ export default class LastfmApiClient extends AbstractApiClient {
user?: string;
declare config: LastfmData;
constructor(name: any, config: Partial<LastfmData>, options = {}) {
constructor(name: any, config: Partial<LastfmData> & {configDir: string}, options = {}) {
super('lastfm', name, config, options);
// @ts-expect-error TS(2339): Property 'redirectUri' does not exist on type '{}'... Remove this comment to see the full error message
const {redirectUri, apiKey, secret, session, configDir} = config;
this.redirectUri = `${redirectUri}?state=${name}`;
if (apiKey === undefined) {
@@ -36,7 +35,7 @@ export default class LastfmApiClient extends AbstractApiClient {
this.client = new LastFm(apiKey as string, secret, session);
}
static formatPlayObj = (obj: TrackObject): PlayObject => {
static formatPlayObj = (obj: TrackObject, options: FormatPlayObjectOptions = {}): PlayObject => {
const {
artist: {
'#text': artists,
+260
View File
@@ -0,0 +1,260 @@
import AbstractApiClient from "./AbstractApiClient.js";
import request, {Request} from 'superagent';
import {ListenBrainzClientData} from "../common/infrastructure/config/client/listenbrainz.js";
import {FormatPlayObjectOptions, PlayObject} from "../common/infrastructure/Atomic.js";
import dayjs from "dayjs";
export interface ArtistMBIDMapping {
artist_credit_name: string
artist_mbid: string
join_phrase: string
}
export interface MinimumTrack {
artist_name: string;
track_name: string;
release_name?: string;
}
export interface AdditionalTrackInfo {
artist_mbids?: string[]
release_mbid?: string
release_group_mbid?: string
recording_mbid?: string
submission_client?: string
submission_client_version?: string
spotify_id?: string
media_player?: string
media_player_version?: string
music_service?: string
music_service_name?: string
origin_url?: string
tags?: string[]
duration?: number
duration_ms?: number
track_mbid?: string
work_mbids?: string[]
}
export interface Track {
artist_name: string;
track_name: string;
release_name?: string;
artist_mbids?: string[];
artist_msid?: string;
recording_mbid?: string;
release_mbid?: string;
release_msid?: string;
tags?: string[];
duration?: number
}
export interface AdditionalTrackInfoResponse extends AdditionalTrackInfo {
recording_msid?: string
}
export interface TrackPayload extends MinimumTrack {
additional_info?: AdditionalTrackInfo
}
export interface ListenPayload {
listened_at: Date | number;
recording_msid?: string;
track_metadata: TrackPayload;
}
export interface SubmitPayload {
listen_type: 'single',
payload: [ListenPayload]
}
export interface TrackResponse extends MinimumTrack {
duration: number
additional_info: AdditionalTrackInfoResponse
mbid_mapping: {
artist_mbids?: string[]
artists?: ArtistMBIDMapping[]
caa_id?: number
caa_release_mbid?: string
recording_mbid?: string
release_mbid?: string
}
}
export interface ListensResponse {
count: number;
listens: ListenResponse[];
}
export interface ListenResponse {
inserted_at: number
listened_at: number;
recording_msid?: string;
track_metadata: TrackResponse;
}
export class ListenbrainzApiClient extends AbstractApiClient {
declare config: ListenBrainzClientData;
url: string;
constructor(name: any, config: ListenBrainzClientData, options = {}) {
super('ListenBrainz', name, config, options);
const {
url = 'https://api.listenbrainz.org/'
} = config;
this.url = url;
}
callApi = async <T>(req: Request, retries = 0): Promise<T> => {
const {
maxRequestRetries = 2,
retryMultiplier = 1.5
} = this.config;
try {
req.set('Authorization', `Token ${this.config.token}`);
return await req as T;
} catch (e) {
const {
message,
} = e;
throw e;
}
}
testConnection = async () => {
try {
const resp = await this.callApi(request.get(this.url))
return true;
} catch (e) {
if(e.status === 410) {
return true;
}
return false;
}
}
testAuth = async () => {
try {
const resp = await this.callApi(request.get(`${this.url}1/validate-token`));
return true;
} catch (e) {
return false;
}
}
getUserListens = async (maxTracks: number, user?: string): Promise<ListensResponse> => {
try {
const resp = await this.callApi(request
.get(`${this.url}1/user/${user ?? this.config.username}/listens`)
// this endpoint can take forever, sometimes, and we want to make sure we timeout in a reasonable amount of time for polling sources to continue trying to scrobble
.timeout({
response: 15000, // wait 15 seconds before timeout if server doesn't response at all
deadline: 30000 // wait 30 seconds overall for request to complete
})
.query({
count: maxTracks
}));
const {body: {payload}} = resp as any;
return payload as ListensResponse;
} catch (e) {
throw e;
}
}
getRecentlyPlayed = async (maxTracks: number, user?: string): Promise<PlayObject[]> => {
try {
const resp = await this.getUserListens(maxTracks, user);
return resp.listens.map(x => ListenbrainzApiClient.listenResponseToPlay(x));
} catch (e) {
this.logger.error(`Error encountered while getting User listens | Error => ${e.message}`);
return [];
}
}
submitListen = async (play: PlayObject) => {
try {
const listenPayload: SubmitPayload = {listen_type: 'single', payload: [ListenbrainzApiClient.playToListenPayload(play)]};
await this.callApi(request.post(`${this.url}1/submit-listens`).type('json').send(listenPayload));
return listenPayload;
} catch (e) {
throw e;
}
}
static playToListenPayload = (play: PlayObject): ListenPayload => {
return {
listened_at: (play.data.playDate ?? dayjs()).unix(),
track_metadata: {
artist_name: play.data.artists[0],
track_name: play.data.track,
additional_info: {
duration: play.data.duration !== undefined ? Math.round(play.data.duration) : undefined
}
}
}
}
static listenResponseToPlay = (listen: ListenResponse): PlayObject => {
const {
listened_at,
recording_msid,
track_metadata: {
track_name,
artist_name,
release_name,
duration,
additional_info: {
recording_msid: aRecordingMsid,
recording_mbid: aRecordingMbid,
duration: aDuration,
duration_ms: aDurationMs,
} = {},
mbid_mapping: {
artists: artistMappings = [],
recording_mbid: mRecordingMbid
} = {}
} = {}
} = listen;
const playId = recording_msid ?? aRecordingMsid;
const trackId = aRecordingMbid ?? mRecordingMbid;
let dur = duration ?? aDuration;
if (dur === undefined && aDurationMs !== undefined) {
dur = Math.round(aDurationMs / 1000);
}
let artists: string[] = [artist_name];
if (artistMappings.length > 0) {
const secondaryArtists = artistMappings.filter(x => x.artist_credit_name !== artist_name);
artists = artists.concat(secondaryArtists.map(x => x.artist_credit_name));
}
return {
data: {
playDate: dayjs.unix(listened_at),
track: track_name,
artists: artists,
album: release_name,
duration: dur
},
meta: {
source: 'listenbrainz',
trackId,
playId
}
}
}
static formatPlayObj = (obj: any, options: FormatPlayObjectOptions): PlayObject => {
return ListenbrainzApiClient.listenResponseToPlay(obj);
}
}
+189 -20
View File
@@ -1,16 +1,24 @@
import dayjs, {Dayjs} from "dayjs";
import {buildTrackString, capitalize, createLabelledLogger, playObjDataMatch} from "../utils.js";
import {
ClientType,
buildTrackString,
capitalize, closePlayDate,
mergeArr,
playObjDataMatch, setIntersection,
truncateStringToLength
} from "../utils.js";
import {
ClientType, FormatPlayObjectOptions,
INITIALIZED,
INITIALIZING,
InitState,
NOT_INITIALIZED,
PlayObject, ScrobbledPlayObject
PlayObject, ScrobbledPlayObject, TrackStringOptions
} from "../common/infrastructure/Atomic.js";
import {Logger} from "winston";
import winston, {Logger} from "winston";
import {CommonClientConfig} from "../common/infrastructure/config/client/index.js";
import {ClientConfig} from "../common/infrastructure/config/client/clients.js";
import {Notifiers} from "../notifier/Notifiers.js";
import {FixedSizeList} from 'fixed-size-list';
export default abstract class AbstractScrobbleClient {
@@ -19,17 +27,19 @@ export default abstract class AbstractScrobbleClient {
#initState: InitState = NOT_INITIALIZED;
protected MAX_STORED_SCROBBLES = 40;
requiresAuth: boolean = false;
requiresAuthInteraction: boolean = false;
authed: boolean = false;
recentScrobbles: PlayObject[] = [];
scrobbledPlayObjs: ScrobbledPlayObject[] = [];
scrobbledPlayObjs: FixedSizeList<ScrobbledPlayObject>;
newestScrobbleTime?: Dayjs
oldestScrobbleTime: Dayjs = dayjs();
oldestScrobbleTime?: Dayjs
tracksScrobbled: number = 0;
lastScrobbleCheck: Dayjs = dayjs();
lastScrobbleCheck: Dayjs = dayjs(0)
refreshEnabled: boolean;
checkExistingScrobbles: boolean;
verboseOptions;
@@ -37,11 +47,16 @@ export default abstract class AbstractScrobbleClient {
config: CommonClientConfig;
logger: Logger;
constructor(type: any, name: any, config: CommonClientConfig) {
notifier: Notifiers;
constructor(type: any, name: any, config: CommonClientConfig, notifier: Notifiers, logger: Logger) {
this.type = type;
this.name = name;
const identifier = `Client ${capitalize(this.type)} - ${name}`;
this.logger = createLabelledLogger(identifier, identifier);
const identifier = `${capitalize(this.type)} - ${name}`;
this.logger = logger.child({labels: [identifier]}, mergeArr);
this.notifier = notifier;
this.scrobbledPlayObjs = new FixedSizeList<ScrobbledPlayObject>(this.MAX_STORED_SCROBBLES);
const {
data: {
@@ -125,29 +140,39 @@ export default abstract class AbstractScrobbleClient {
return this.lastScrobbleCheck;
}
formatPlayObj = (obj: any) => {
formatPlayObj = (obj: any, options: FormatPlayObjectOptions = {}) => {
this.logger.warn('formatPlayObj should be defined by concrete class!');
return obj;
}
// time frame is valid as long as the play date for the source track is newer than the oldest play time from the scrobble client
// ...this is assuming the scrobble client is returning "most recent" scrobbles
timeFrameIsValid = (playObj: any, log = false) => {
timeFrameIsValid = (playObj: PlayObject) => {
if(this.oldestScrobbleTime === undefined) {
return [true, ''];
}
const {
data: {
// @ts-expect-error TS(2525): Initializer provides no value for this binding ele... Remove this comment to see the full error message
playDate,
} = {},
} = playObj;
const validTime = playDate.isAfter(this.oldestScrobbleTime);
if (log && !validTime) {
this.logger.debug(`${buildTrackString(playObj)} was in an invalid time frame (played before the oldest scrobble found)`);
let log = '';
if (!validTime) {
const dur = dayjs.duration(Math.abs(playDate.diff(this.oldestScrobbleTime))).humanize(false);
log = `occurred ${dur} before the oldest scrobble returned by this client (${this.oldestScrobbleTime.format()})`;
}
return validTime;
return [validTime, log]
}
addScrobbledTrack = (playObj: any, scrobbleResp: any) => {
this.scrobbledPlayObjs.push({play: playObj, scrobble: this.formatPlayObj(scrobbleResp)});
addScrobbledTrack = (playObj: PlayObject, scrobbledPlay: PlayObject) => {
this.scrobbledPlayObjs.add({play: playObj, scrobble: scrobbledPlay});
}
filterScrobbledTracks = () => {
this.scrobbledPlayObjs = new FixedSizeList<ScrobbledPlayObject>(this.MAX_STORED_SCROBBLES, this.scrobbledPlayObjs.data.filter(x => this.timeFrameIsValid(x.play)[0])) ;
}
cleanSourceSearchTitle = (playObj: PlayObject) => {
@@ -157,7 +182,7 @@ export default abstract class AbstractScrobbleClient {
} = {},
} = playObj;
return track;
return track.toLocaleLowerCase().trim();
};
findExistingSubmittedPlayObj = (playObj: PlayObject): ([undefined, undefined] | [ScrobbledPlayObject, ScrobbledPlayObject[]]) => {
@@ -170,7 +195,7 @@ export default abstract class AbstractScrobbleClient {
} = {}
} = playObj;
const dtInvariantMatches = this.scrobbledPlayObjs.filter(x => playObjDataMatch(playObj, x.play));
const dtInvariantMatches = this.scrobbledPlayObjs.data.filter(x => playObjDataMatch(playObj, x.play));
if (dtInvariantMatches.length === 0) {
return [undefined, undefined];
@@ -196,4 +221,148 @@ export default abstract class AbstractScrobbleClient {
return [matchPlayDate, dtInvariantMatches];
}
protected compareExistingScrobbleTime = (existing: PlayObject, candidate: PlayObject): [boolean, boolean?] => {
let closeTime = closePlayDate(existing, candidate);
let fuzzyTime = false;
if(!closeTime) {
fuzzyTime = closePlayDate(existing, candidate, {fuzzyDuration: true});
}
return [closeTime, fuzzyTime];
}
protected compareExistingScrobbleTitle = (existing: PlayObject, candidate: PlayObject): number => {
const {
data: {
track: scrobbleTitle,
} = {},
} = existing;
let cleanSourceTitle = this.cleanSourceSearchTitle(candidate);
let titleMatch;
const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
// because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
// and compare against length of scrobble title
const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter((x: any) => x !== ''));
const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
titleMatch = commonTerms.size / sourceTitleTerms.size;
return titleMatch;
}
protected compareExistingScrobbleArtist = (existing: PlayObject, candidate: PlayObject): number => {
const {
data: {
artists: sourceArtists = [],
} = {},
} = candidate;
const {
data: {
artists = [],
} = {},
} = candidate;
let artistMatch;
const lowerSourceArtists = sourceArtists.map((x: any) => x.toLocaleLowerCase());
const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
return artistMatch;
}
existingScrobble = async (playObj: PlayObject) => {
const tr = truncateStringToLength(27);
const scoreTrackOpts: TrackStringOptions = {include: ['track', 'time'], transformers: {track: (t: any) => tr(t).padEnd(30)}};
// return early if we don't care about checking existing
if (false === this.checkExistingScrobbles) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
}
return undefined;
}
let existingScrobble;
let closestMatch: {score: number, breakdowns: string[], scrobble?: PlayObject} = {score: 0, breakdowns: ['None']};
// then check if we have already recorded this
const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
// if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
if (existingExactSubmitted !== undefined) {
existingScrobble = existingExactSubmitted.scrobble;
closestMatch = {
score: 1,
breakdowns: ['Exact Match found in previously successfully scrobbled']
}
}
// if not though then we need to check recent scrobbles from scrobble api.
// this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
if (existingScrobble === undefined) {
// if no recent scrobbles found then assume we haven't submitted it
// (either user doesnt want to check history or there is no history to check!)
if (this.recentScrobbles.length === 0) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
}
return undefined;
}
// we have have found an existing submission but without an exact date
// in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
// clean source title so it matches title from the scrobble api response as closely as we can get it
let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
existingScrobble = this.recentScrobbles.find((x) => {
const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
const [closeTime, fuzzyTime = false] = this.compareExistingScrobbleTime(x, playObj);
const titleMatch = this.compareExistingScrobbleTitle(x, playObj);
const artistMatch = this.compareExistingScrobbleArtist(x, playObj);
const artistScore = .2 * artistMatch;
const titleScore = .3 * titleMatch;
const timeScore = .5 * (closeTime ? 1 : (fuzzyTime ? 0.5 : 0));
const referenceScore = .5 * (referenceMatch ? 1 : 0);
const score = artistScore + titleScore + timeScore;
let scoreBreakdowns = [
`Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
`Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
`Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
`Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
`Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
];
const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
const scoreInfo = {
score,
scrobble: x,
breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
}
if (closestMatch.score <= score && score > 0) {
closestMatch = scoreInfo
}
return score >= .7;
});
}
if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
}
return existingScrobble;
}
}
+22 -145
View File
@@ -2,16 +2,23 @@ import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
import dayjs from 'dayjs';
import {
buildTrackString,
buildTrackString, capitalize,
playObjDataMatch, removeUndefinedKeys,
setIntersection, sleep,
sortByPlayDate,
sortByOldestPlayDate,
truncateStringToLength,
} from "../utils.js";
import LastfmApiClient from "../apis/LastfmApiClient.js";
import {INITIALIZING, PlayObject, TrackStringOptions} from "../common/infrastructure/Atomic.js";
import {
FormatPlayObjectOptions,
INITIALIZING,
PlayObject, ScrobbledPlayObject,
TrackStringOptions
} from "../common/infrastructure/Atomic.js";
import {LastfmClientConfig} from "../common/infrastructure/config/client/lastfm.js";
import {TrackScrobbleResponse, UserGetRecentTracksResponse} from "lastfm-node-client";
import {Notifiers} from "../notifier/Notifiers.js";
import {Logger} from "winston";
export default class LastfmScrobbler extends AbstractScrobbleClient {
@@ -21,12 +28,13 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
declare config: LastfmClientConfig;
constructor(name: any, config: LastfmClientConfig, options = {}) {
super('lastfm', name, config);
constructor(name: any, config: LastfmClientConfig, options = {}, notifier: Notifiers, logger: Logger) {
super('lastfm', name, config, notifier, logger);
// @ts-ignore
this.api = new LastfmApiClient(name, config.data, options)
}
formatPlayObj = (obj: any) => LastfmApiClient.formatPlayObj(obj);
formatPlayObj = (obj: any, options: FormatPlayObjectOptions = {}) => LastfmApiClient.formatPlayObj(obj, options);
initialize = async () => {
// @ts-expect-error TS(2322): Type 'number' is not assignable to type 'boolean'.
@@ -49,7 +57,7 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
refreshScrobbles = async () => {
if (this.refreshEnabled) {
this.logger.debug('Refreshing recent scrobbles');
const resp = await this.api.callApi<UserGetRecentTracksResponse>((client: any) => client.userGetRecentTracks({user: this.api.user, limit: 20, extended: true}));
const resp = await this.api.callApi<UserGetRecentTracksResponse>((client: any) => client.userGetRecentTracks({user: this.api.user, limit: this.MAX_STORED_SCROBBLES, extended: true}));
const {
recenttracks: {
track: list = [],
@@ -85,14 +93,15 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
this.logger.debug(x);
return acc;
}
}, []).sort(sortByPlayDate);
}, []).sort(sortByOldestPlayDate);
this.logger.debug(`Found ${this.recentScrobbles.length} recent scrobbles`);
if (this.recentScrobbles.length > 0) {
const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
this.newestScrobbleTime = newestScrobbleTime;
this.oldestScrobbleTime = oldestScrobbleTime;
this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
this.filterScrobbledTracks();
}
}
this.lastScrobbleCheck = dayjs();
@@ -108,141 +117,7 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
}
alreadyScrobbled = async (playObj: PlayObject, log = false) => {
return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
}
existingScrobble = (playObj: PlayObject, logMatch = false) => {
const tr = truncateStringToLength(27);
const scoreTrackOpts: TrackStringOptions = {include: ['track', 'time'], transformers: {track: (t: any) => tr(t).padEnd(30)}};
// return early if we don't care about checking existing
if (false === this.checkExistingScrobbles) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
}
return undefined;
}
let existingScrobble;
let closestMatch: {score: number, breakdowns: string[], scrobble?: PlayObject} = {score: 0, breakdowns: ['None']};
// then check if we have already recorded this
const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
// if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
if (existingExactSubmitted !== undefined) {
existingScrobble = existingExactSubmitted.scrobble;
closestMatch = {
score: 1,
breakdowns: ['Exact Match found in previously successfully scrobbled']
}
}
// if not though then we need to check recent scrobbles from scrobble api.
// this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
// if no recent scrobbles found then assume we haven't submitted it
// (either user doesnt want to check history or there is no history to check!)
if (this.recentScrobbles.length === 0) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
}
return undefined;
}
if (existingScrobble === undefined) {
// we have have found an existing submission but without an exact date
// in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
const {
data: {
artists: sourceArtists = [],
playDate
} = {},
meta: {
trackLength,
source,
} = {},
} = playObj;
// clean source title so it matches title from the scrobble api response as closely as we can get it
let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
existingScrobble = this.recentScrobbles.find((x) => {
const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
let closeTime = false;
// check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
let scrobblePlayStartDiff;
if (scrobblePlayDiff <= playDiffThreshold) {
//this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
closeTime = true;
}
// also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
if (closeTime === false && trackLength !== undefined) {
scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
if (scrobblePlayStartDiff <= playDiffThreshold) {
//this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
closeTime = true;
}
}
let titleMatch;
const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
// because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
// and compare against length of scrobble title
const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter((x: any) => x !== ''));
const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
titleMatch = commonTerms.size / sourceTitleTerms.size;
let artistMatch;
const lowerSourceArtists = sourceArtists.map((x: any) => x.toLocaleLowerCase());
const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
const artistScore = .2 * artistMatch;
const titleScore = .3 * titleMatch;
const timeScore = .5 * (closeTime ? 1 : 0);
const referenceScore = .5 * (referenceMatch ? 1 : 0);
const score = artistScore + titleScore + timeScore;
let scoreBreakdowns = [
`Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
`Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
`Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
`Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
`Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
];
const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
const scoreInfo = {
score,
scrobble: x,
breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
}
if (closestMatch.score <= score && score > 0) {
closestMatch = scoreInfo
}
return score >= .7;
});
}
if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
}
return existingScrobble;
return await this.existingScrobble(playObj) !== undefined;
}
scrobble = async (playObj: PlayObject) => {
@@ -305,19 +180,21 @@ export default class LastfmScrobbler extends AbstractScrobbleClient {
this.initialized = false;
throw new Error('Service reported daily scrobble limit exceeded! 😬 Disabling client');
}
this.addScrobbledTrack(playObj, {...rest, date: { uts: timestamp}, name: trackName});
this.addScrobbledTrack(playObj, this.formatPlayObj({...rest, date: { uts: timestamp}, name: trackName}));
if (newFromSource) {
this.logger.info(`Scrobbled (New) => (${source}) ${buildTrackString(playObj)}`);
} else {
this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
}
if(ignored > 0) {
await this.notifier.notify({title: `Client - ${capitalize(this.type)} - ${this.name} - Scrobble Error`, message: `Failed to scrobble => ${buildTrackString(playObj)} | Error: Service ignored this scrobble 😬 => (Code ${ignoreCode}) ${(ignoreMsg === '' ? '(No error message returned)' : ignoreMsg)}`, priority: 'warn'});
this.logger.warn(`Service ignored this scrobble 😬 => (Code ${ignoreCode}) ${(ignoreMsg === '' ? '(No error message returned)' : ignoreMsg)} -- See https://www.last.fm/api/errorcodes for more information`, {payload: scrobblePayload});
}
// last fm has rate limits but i can't find a specific example of what that limit is. going to default to 1 scrobble/sec to be safe
await sleep(1000);
} catch (e) {
await this.notifier.notify({title: `Client - ${capitalize(this.type)} - ${this.name} - Scrobble Error`, message: `Failed to scrobble => ${buildTrackString(playObj)} | Error: ${e.message}`, priority: 'error'});
this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj), payload: scrobblePayload});
throw e;
} finally {
+130
View File
@@ -0,0 +1,130 @@
import dayjs from 'dayjs';
import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
import {
buildTrackString, capitalize,
playObjDataMatch, removeUndefinedKeys,
setIntersection, sleep,
sortByOldestPlayDate,
truncateStringToLength,
} from "../utils.js";
import LastfmApiClient from "../apis/LastfmApiClient.js";
import {
FormatPlayObjectOptions,
INITIALIZING,
PlayObject,
TrackStringOptions
} from "../common/infrastructure/Atomic.js";
import {Notifiers} from "../notifier/Notifiers.js";
import {Logger} from "winston";
import {ListenBrainzClientConfig} from "../common/infrastructure/config/client/listenbrainz.js";
import {ListenbrainzApiClient} from "../apis/ListenbrainzApiClient.js";
export default class ListenbrainzScrobbler extends AbstractScrobbleClient {
api: ListenbrainzApiClient;
requiresAuth = true;
requiresAuthInteraction = false;
declare config: ListenBrainzClientConfig;
constructor(name: any, config: ListenBrainzClientConfig, options = {}, notifier: Notifiers, logger: Logger) {
super('listenbrainz', name, config, notifier, logger);
this.api = new ListenbrainzApiClient(name, config.data);
}
formatPlayObj = (obj: any, options: FormatPlayObjectOptions = {}) => ListenbrainzApiClient.formatPlayObj(obj, options);
initialize = async () => {
// @ts-expect-error TS(2322): Type 'number' is not assignable to type 'boolean'.
this.initialized = INITIALIZING;
if(this.config.data.token === undefined) {
this.logger.error('Must provide a User Token');
this.initialized = false;
} else {
try {
await this.api.testConnection();
this.initialized = true;
} catch (e) {
this.logger.error(e);
this.initialized = false;
}
}
return this.initialized;
}
testAuth = async () => {
try {
this.authed = await this.api.testAuth();
} catch (e) {
this.logger.error('Could not successfully communicate with Listenbrainz API');
this.logger.error(e);
this.authed = false;
}
return this.authed;
}
refreshScrobbles = async () => {
if (this.refreshEnabled) {
this.logger.debug('Refreshing recent scrobbles');
const resp = await this.api.getRecentlyPlayed(this.MAX_STORED_SCROBBLES);
this.logger.debug(`Found ${resp.length} recent scrobbles`);
this.recentScrobbles = resp.sort(sortByOldestPlayDate);
if (this.recentScrobbles.length > 0) {
const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
this.newestScrobbleTime = newestScrobbleTime;
this.oldestScrobbleTime = oldestScrobbleTime;
this.filterScrobbledTracks();
}
}
this.lastScrobbleCheck = dayjs();
}
alreadyScrobbled = async (playObj: PlayObject, log = false) => {
return await this.existingScrobble(playObj) !== undefined;
}
scrobble = async (playObj: PlayObject) => {
const {
meta: {
source,
newFromSource = false,
} = {}
} = playObj;
let rawPayload = {listen_type: 'single', payload: [ListenbrainzApiClient.playToListenPayload(playObj)]};
try {
const resp = await this.api.submitListen(playObj);
rawPayload = resp;
this.addScrobbledTrack(playObj, playObj);
if (newFromSource) {
this.logger.info(`Scrobbled (New) => (${source}) ${buildTrackString(playObj)}`);
} else {
this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
}
// last fm has rate limits but i can't find a specific example of what that limit is. going to default to 1 scrobble/sec to be safe
await sleep(1000);
} catch (e) {
let message = e.message;
if(e.response !== undefined) {
if(e.response.body !== undefined) {
message = e.response.body.messsage;
} else if(e.response.text !== undefined) {
message = e.response.text;
}
}
await this.notifier.notify({title: `Client - ${capitalize(this.type)} - ${this.name} - Scrobble Error`, message: `Failed to scrobble => ${buildTrackString(playObj)} | Error: ${message}`, priority: 'error'});
this.logger.error(`Failed to scrobble => ${message}`, {payload: rawPayload});
throw e;
} finally {
this.logger.debug(`Raw Payload:`, {rawPayload});
}
return true;
}
}
+21 -147
View File
@@ -7,11 +7,12 @@ import {
playObjDataMatch,
setIntersection,
sleep,
sortByPlayDate,
sortByOldestPlayDate,
truncateStringToLength,
parseRetryAfterSecsFromObj
parseRetryAfterSecsFromObj, capitalize, closePlayDate
} from "../utils.js";
import {
FormatPlayObjectOptions,
INITIALIZING,
MalojaScrobbleData,
MalojaScrobbleRequestData,
@@ -23,6 +24,8 @@ import {
TrackStringOptions
} from "../common/infrastructure/Atomic.js";
import {MalojaClientConfig} from "../common/infrastructure/config/client/maloja.js";
import {Notifiers} from "../notifier/Notifiers.js";
import {Logger} from "winston";
const feat = ["ft.", "ft", "feat.", "feat", "featuring", "Ft.", "Ft", "Feat.", "Feat", "Featuring"];
@@ -34,8 +37,8 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
declare config: MalojaClientConfig
constructor(name: any, config: MalojaClientConfig) {
super('maloja', name, config);
constructor(name: any, config: MalojaClientConfig, notifier: Notifiers, logger: Logger) {
super('maloja', name, config, notifier, logger);
const {url, apiKey} = config.data;
if (apiKey === undefined) {
this.logger.warn("'apiKey' not found in config! Client will most likely fail when trying to scrobble");
@@ -45,13 +48,15 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
}
}
static formatPlayObj(obj: MalojaScrobbleData, serverVersion = undefined): PlayObject {
static formatPlayObj(obj: MalojaScrobbleData, options: FormatPlayObjectOptions = {}): PlayObject {
let artists,
title,
album,
duration,
time;
const {serverVersion} = options;
if(serverVersion === undefined || compareVersions(serverVersion, '3.0.0') >= 0) {
// scrobble data structure changed for v3
const {
@@ -73,7 +78,7 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
artists = mArtists;
time = mTime;
title = mTitle;
duration = mDuration;
duration = mLength;
album = mAlbum;
} else {
// scrobble data structure for v2 and below
@@ -114,7 +119,7 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
}
}
formatPlayObj = (obj: any) => MalojaScrobbler.formatPlayObj(obj, this.serverVersion);
formatPlayObj = (obj: any, options: FormatPlayObjectOptions = {}) => MalojaScrobbler.formatPlayObj(obj, {serverVersion: this.serverVersion});
callApi = async (req: any, retries = 0) => {
const {
@@ -304,20 +309,21 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
if (this.refreshEnabled) {
this.logger.debug('Refreshing recent scrobbles');
const {url} = this.config.data;
const resp = await this.callApi(request.get(`${url}/apis/mlj_1/scrobbles?max=20`));
const resp = await this.callApi(request.get(`${url}/apis/mlj_1/scrobbles?max=${this.MAX_STORED_SCROBBLES}`));
const {
body: {
list = [],
} = {},
} = resp;
this.recentScrobbles = list.map((x: any) => this.formatPlayObj(x)).sort(sortByPlayDate);
this.logger.debug(`Found ${list.length} recent scrobbles`);
this.recentScrobbles = list.map((x: any) => this.formatPlayObj(x)).sort(sortByOldestPlayDate);
if (this.recentScrobbles.length > 0) {
const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
this.newestScrobbleTime = newestScrobbleTime;
this.oldestScrobbleTime = oldestScrobbleTime;
this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
this.filterScrobbledTracks();
}
}
this.lastScrobbleCheck = dayjs();
@@ -347,141 +353,7 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
}
alreadyScrobbled = async (playObj: any, log = false) => {
return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
}
existingScrobble = (playObj: PlayObject, logMatch = false) => {
const tr = truncateStringToLength(27);
const scoreTrackOpts: TrackStringOptions = {include: ['track', 'time'], transformers: {track: (t: any) => tr(t).padEnd(30)}};
// return early if we don't care about checking existing
if (false === this.checkExistingScrobbles) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
}
return undefined;
}
let existingScrobble;
let closestMatch: {score: number, breakdowns: string[], scrobble?: PlayObject} = {score: 0, breakdowns: ['None']};
// then check if we have already recorded this
const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
// if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
if (existingExactSubmitted !== undefined) {
existingScrobble = existingExactSubmitted.scrobble;
closestMatch = {
score: 1,
breakdowns: ['Exact Match found in previously successfully scrobbled']
}
}
// if not though then we need to check recent scrobbles from scrobble api.
// this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
// if no recent scrobbles found then assume we haven't submitted it
// (either user doesnt want to check history or there is no history to check!)
if (this.recentScrobbles.length === 0) {
if (this.verboseOptions.match.onNoMatch) {
this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
}
return undefined;
}
if (existingScrobble === undefined) {
// we have have found an existing submission but without an exact date
// in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
const {
data: {
artists: sourceArtists = [],
playDate
} = {},
meta: {
trackLength,
source,
} = {},
} = playObj;
// clean source title so it matches title from the scrobble api response as closely as we can get it
let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
existingScrobble = this.recentScrobbles.find((x) => {
const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
let closeTime = false;
// check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
let scrobblePlayStartDiff;
if (scrobblePlayDiff <= playDiffThreshold) {
//this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
closeTime = true;
}
// also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
if (closeTime === false && trackLength !== undefined) {
scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
if (scrobblePlayStartDiff <= playDiffThreshold) {
//this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
closeTime = true;
}
}
let titleMatch;
const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
// because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
// and compare against length of scrobble title
const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter((x: any) => x !== ''));
const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
titleMatch = commonTerms.size / sourceTitleTerms.size;
let artistMatch;
const lowerSourceArtists = sourceArtists.map((x: any) => x.toLocaleLowerCase());
const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
const artistScore = .2 * artistMatch;
const titleScore = .3 * titleMatch;
const timeScore = .5 * (closeTime ? 1 : 0);
const referenceScore = .5 * (referenceMatch ? 1 : 0);
const score = artistScore + titleScore + timeScore;
let scoreBreakdowns = [
`Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
`Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
`Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
`Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
`Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
];
const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
const scoreInfo = {
score,
scrobble: x,
breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
}
if (closestMatch.score <= score && score > 0) {
closestMatch = scoreInfo
}
return score >= .7;
});
}
if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
}
return existingScrobble;
return await this.existingScrobble(playObj) !== undefined;
}
scrobble = async (playObj: PlayObject) => {
@@ -564,16 +436,18 @@ export default class MalojaScrobbler extends AbstractScrobbleClient {
} = {}} = response;
scrobbleResponse = {...rest, album: mAlbum, time: mTime, duration: mDuration};
}
this.addScrobbledTrack(playObj, scrobbleResponse);
this.addScrobbledTrack(playObj, this.formatPlayObj(scrobbleResponse));
if (newFromSource) {
this.logger.info(`Scrobbled (New) => (${source}) ${buildTrackString(playObj)}`);
} else {
this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
}
this.logger.debug('Payload:', scrobbleData);
} catch (e) {
await this.notifier.notify({title: `Client - ${capitalize(this.type)} - ${this.name} - Scrobble Error`, message: `Failed to scrobble => ${buildTrackString(playObj)} | Error: ${e.message}`, priority: 'error'});
this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj), payload: scrobbleData});
throw e;
} finally {
this.logger.debug('Raw Payload:', scrobbleData);
}
return true;
+94 -27
View File
@@ -1,7 +1,8 @@
import dayjs, {Dayjs} from "dayjs";
import {
buildTrackString,
createAjvFactory,
createLabelledLogger,
mergeArr,
playObjDataMatch,
readJson,
returnDuplicateStrings, validateJson
@@ -15,6 +16,13 @@ import * as clientSchema from '../common/schema/client.json';
import {ClientAIOConfig, ClientConfig} from "../common/infrastructure/config/client/clients.js";
import {MalojaClientConfig} from "../common/infrastructure/config/client/maloja.js";
import {LastfmClientConfig} from "../common/infrastructure/config/client/lastfm.js";
import {Notifiers} from "../notifier/Notifiers.js";
import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
import {EventEmitter} from "events";
import winston from "winston";
import ListenbrainzScrobbler from "./ListenbrainzScrobbler.js";
import {ListenBrainzClientConfig} from "../common/infrastructure/config/client/listenbrainz.js";
import {ErrorWithCause} from "pony-cause";
type groupedNamedConfigs = {[key: string]: ParsedConfig[]};
@@ -27,9 +35,19 @@ export default class ScrobbleClients {
logger;
configDir;
constructor(configDir: any) {
emitter: EventEmitter;
sourceEmitter: EventEmitter;
constructor(emitter: EventEmitter, sourceEmitter: EventEmitter, configDir: any) {
this.emitter = emitter;
this.sourceEmitter = sourceEmitter;
this.configDir = configDir;
this.logger = createLabelledLogger('scrobblers', 'Scrobblers');
this.logger = winston.loggers.get('app').child({labels: ['Scrobblers']}, mergeArr);
this.sourceEmitter.on('scrobble', async (payload: { data: (PlayObject | PlayObject[]), options: { forceRefresh?: boolean, checkTime?: Dayjs, scrobbleTo?: string[], scrobbleFrom?: string } }) => {
await this.scrobble(payload.data, payload.options);
});
}
getByName = (name: any) => {
@@ -40,7 +58,31 @@ export default class ScrobbleClients {
return this.clients.filter(x => x.type === type);
}
buildClientsFromConfig = async () => {
async getStatusSummary(type?: string, name?: string): Promise<[boolean, string[]]> {
let clients: AbstractScrobbleClient[];
const messages: string[] = [];
let clientsReady = true;
if(type !== undefined) {
clients = this.getByType(type);
} else if(name !== undefined) {
clients = [this.getByName(name)];
} else {
clients = this.clients;
}
for(const client of clients) {
if(!await client.isReady()) {
clientsReady = false;
messages.push(`Client ${client.type} - ${client.name} is not ready.`);
}
}
return [clientsReady, messages];
}
buildClientsFromConfig = async (notifier: Notifiers) => {
let configs: ParsedConfig[] = [];
let configFile;
@@ -89,7 +131,7 @@ export default class ScrobbleClients {
if (url !== undefined || apiKey !== undefined) {
configs.push({
type: 'maloja',
name: 'unnamed',
name: 'unnamed-mlj',
source: 'ENV',
mode: 'single',
configureAs: 'client',
@@ -111,7 +153,7 @@ export default class ScrobbleClients {
if (!Object.values(lfm).every(x => x === undefined)) {
configs.push({
type: 'lastfm',
name: 'unnamed',
name: 'unnamed-lfm',
source: 'ENV',
mode: 'single',
configureAs: 'client',
@@ -120,6 +162,24 @@ export default class ScrobbleClients {
})
}
break;
case 'listenbrainz':
const lz = {
url: process.env.LZ_URL,
token: process.env.LZ_TOKEN,
username: process.env.LZ_USER
};
if (!Object.values(lz).every(x => x === undefined)) {
configs.push({
type: 'listenbrainz',
name: 'unnamed-lz',
source: 'ENV',
mode: 'single',
configureAs: 'client',
// @ts-ignore
data: lz
})
}
break;
default:
break;
}
@@ -218,7 +278,7 @@ ${sources.join('\n')}`);
}));
for (const c of finalConfigs) {
try {
await this.addClient(c, clientDefaults);
await this.addClient(c, clientDefaults, notifier);
} catch(e) {
this.logger.error(`Client ${c.name} was not added because it had unrecoverable errors`);
this.logger.error(e);
@@ -226,7 +286,7 @@ ${sources.join('\n')}`);
}
}
addClient = async (clientConfig: ParsedConfig, defaults = {}) => {
addClient = async (clientConfig: ParsedConfig, defaults = {}, notifier: Notifiers) => {
/* const isValidConfig = isValidConfigStructure(clientConfig, {name: true, data: true, type: true});
if (isValidConfig !== true) {
throw new Error(`Config object from ${clientConfig.source || 'unknown'} with name [${clientConfig.name || 'unnamed'}] of type [${clientConfig.type || 'unknown'}] has errors: ${isValidConfig.join(' | ')}`)
@@ -235,13 +295,16 @@ ${sources.join('\n')}`);
// add defaults
const data = {...defaults, ...d};
let newClient;
this.logger.debug(`(${name}) Constructing ${type} client...`);
this.logger.debug(`Constructing ${type} (${name}) client...`);
switch (type) {
case 'maloja':
newClient = new MalojaScrobbler(name, ({...clientConfig, data} as unknown as MalojaClientConfig));
newClient = new MalojaScrobbler(name, ({...clientConfig, data} as unknown as MalojaClientConfig), notifier, this.logger);
break;
case 'lastfm':
newClient = new LastfmScrobbler(name, {...clientConfig, data: {configDir: this.configDir, ...data} } as unknown as LastfmClientConfig );
newClient = new LastfmScrobbler(name, {...clientConfig, data: {configDir: this.configDir, ...data} } as unknown as LastfmClientConfig, {}, notifier, this.logger);
break;
case 'listenbrainz':
newClient = new ListenbrainzScrobbler(name, {...clientConfig, data: {configDir: this.configDir, ...data} } as unknown as ListenBrainzClientConfig, {}, notifier, this.logger);
break;
default:
break;
@@ -252,15 +315,15 @@ ${sources.join('\n')}`);
throw new Error(`Client of type ${type} was not recognized??`);
}
if(newClient.initialized === false) {
this.logger.debug(`(${name}) Attempting ${type} initialization...`);
this.logger.debug(`Attempting ${type} (${name}) initialization...`);
if ((await newClient.initialize()) === false) {
this.logger.error(`(${name}) ${type} client failed to initialize. Client needs to be successfully initialized before scrobbling.`);
this.logger.error(`${type} (${name}) client failed to initialize. Client needs to be successfully initialized before scrobbling.`);
} else {
this.logger.info(`(${name}) ${type} client initialized`);
this.logger.info(`${type} (${name}) client initialized`);
}
}
if(newClient.requiresAuth && !newClient.authed) {
this.logger.debug(`(${name}) Checking ${type} client auth...`);
this.logger.debug(`Checking ${type} (${name}) client auth...`);
let success;
try {
success = await newClient.testAuth();
@@ -268,9 +331,9 @@ ${sources.join('\n')}`);
success = false;
}
if(!success) {
this.logger.warn(`(${name}) ${type} client auth failed.`);
this.logger.warn(`${type} (${name}) client auth failed.`);
} else {
this.logger.info(`(${name}) ${type} client auth OK`);
this.logger.info(`${type} (${name}) client auth OK`);
}
}
this.clients.push(newClient);
@@ -298,32 +361,32 @@ ${sources.join('\n')}`);
for (const client of this.clients) {
if (scrobbleTo.length > 0 && !scrobbleTo.includes(client.name)) {
this.logger.debug(`Client '${client.name}' was filtered out by '${scrobbleFrom}'`);
client.logger.debug(`Client was filtered out by Source '${scrobbleFrom}'`);
continue;
}
if(!client.initialized) {
if(client.initializing) {
this.logger.warn(`Cannot scrobble to Client '${client.name}' because it is still initializing`);
client.logger.warn(`Cannot scrobble because it is still initializing`);
continue;
}
if(!(await client.initialize())) {
this.logger.warn(`Cannot scrobble to Client '${client.name}' because it could not be initialized`);
client.logger.warn(`Cannot scrobble because it could not be initialized`);
continue;
}
}
if(client.requiresAuth && !client.authed) {
if (client.requiresAuthInteraction) {
this.logger.warn(`Cannot scrobble to Client '${client.name}' because user interaction is required for authentication`);
client.logger.warn(`Cannot scrobble because user interaction is required for authentication`);
continue;
} else if (!(await client.testAuth())) {
this.logger.warn(`Cannot scrobble to Client '${client.name}' because auth test failed`);
client.logger.warn(`Cannot scrobble because auth test failed`);
continue;
}
}
if(!(await client.isReady())) {
this.logger.warn(`Cannot scrobble to Client '${client.name}' because it is not ready`);
client.logger.warn(`Cannot scrobble because it is not ready`);
continue;
}
@@ -331,7 +394,7 @@ ${sources.join('\n')}`);
try {
await client.refreshScrobbles();
} catch(e) {
this.logger.error(`Encountered error while refreshing scrobbles for ${client.name}`);
client.logger.error(`Encountered error while refreshing scrobbles`);
this.logger.error(e);
}
}
@@ -342,7 +405,8 @@ ${sources.join('\n')}`);
newFromSource = false,
} = {}
} = playObj;
if (client.timeFrameIsValid(playObj, newFromSource) && !(await client.alreadyScrobbled(playObj, newFromSource))) {
const [timeFrameValid, timeFrameValidLog] = client.timeFrameIsValid(playObj);
if (timeFrameValid && !(await client.alreadyScrobbled(playObj))) {
await client.scrobble(playObj)
client.tracksScrobbled++;
// since this is what we return to the source only add to tracksScrobbled if not already in array
@@ -350,10 +414,13 @@ ${sources.join('\n')}`);
if(!tracksScrobbled.some(x => playObjDataMatch(x, playObj) && x.data.playDate === playObj.data.playDate)) {
tracksScrobbled.push(playObj);
}
} else {
if(!timeFrameValid) {
client.logger.debug(`Will not scrobble ${buildTrackString(playObj)} from Source '${scrobbleFrom}' because it ${timeFrameValidLog}`);
}
}
} catch(e) {
this.logger.error(`Encountered error while in scrobble loop for ${client.name}`);
this.logger.error(e);
client.logger.error(new ErrorWithCause(`Encountered error while in scrobble loop`, {cause: e}));
// for now just stop scrobbling plays for this client and move on. the client should deal with logging the issue
if(e.continueScrobbling !== true) {
break;
+1 -1
View File
@@ -1,5 +1,5 @@
import * as path from 'path';
import {fileURLToPath} from "url";
//import {fileURLToPath} from "url";
//const __filename = fileURLToPath(import.meta.url);
//const __dirname = path.dirname(__filename);
+108 -7
View File
@@ -1,10 +1,15 @@
import {Dayjs} from "dayjs";
import {FixedSizeList} from 'fixed-size-list';
import {MESSAGE} from 'triple-beam';
import {Logger} from "winston";
export type SourceType = 'spotify' | 'plex' | 'tautulli' | 'subsonic' | 'jellyfin' | 'lastfm' | 'deezer';
export const sourceTypes: SourceType[] = ['spotify', 'plex', 'tautulli', 'subsonic', 'jellyfin', 'lastfm', 'deezer'];
export type SourceType = 'spotify' | 'plex' | 'tautulli' | 'subsonic' | 'jellyfin' | 'lastfm' | 'deezer' | 'ytmusic' | 'mpris' | 'mopidy' | 'listenbrainz' | 'jriver' | 'kodi';
export const sourceTypes: SourceType[] = ['spotify', 'plex', 'tautulli', 'subsonic', 'jellyfin', 'lastfm', 'deezer', 'ytmusic', 'mpris', 'mopidy', 'listenbrainz', 'jriver', 'kodi'];
export type ClientType = 'maloja' | 'lastfm';
export const clientTypes: ClientType[] = ['maloja', 'lastfm'];
export const lowGranularitySources: SourceType[] = ['subsonic','ytmusic'];
export type ClientType = 'maloja' | 'lastfm' | 'listenbrainz';
export const clientTypes: ClientType[] = ['maloja', 'lastfm', 'listenbrainz'];
export type InitState = 0 | 1 | 2;
export const NOT_INITIALIZED: InitState = 0;
@@ -21,6 +26,8 @@ export const readyStates: ReadyState[] = [NOT_READY, GETTING_READY, READY];
export interface InternalConfig {
localUrl: string
configDir: string
logger: Logger
}
export interface ConfigMeta {
@@ -33,23 +40,49 @@ export interface PlayData {
artists?: string[]
album?: string
track?: string
/**
* The length of the track, in seconds
* */
duration?: number
/**
* The date the track was played at
* */
playDate?: Dayjs
}
export interface PlayMeta {
source?: string
sourceId?: string
/**
* Specifies from what facet/data from the source this play was parsed from IE history, now playing, etc...
* */
parsedFrom?: string
/**
* Unique ID for this track, given by the Source
* */
trackId?: string
/**
* Atomic ID for this instance of played tracked IE a unique ID for "this track played at this time"
* */
playId?: string
newFromSource?: boolean
url?: {
web: string
[key: string]: string
}
trackLength?: number
user?: string
mediaType?: string
server?: string
library?: string
/**
* The position the "player" is at in the track at the time the play was reported, in seconds
* */
trackProgressPosition?: number
/**
* A unique identifier for the device playing this track
* */
deviceId?: string
[key: string]: any
}
@@ -59,8 +92,25 @@ export interface PlayObject {
meta: PlayMeta
}
export interface FormatPlayObjectOptions {
newFromSource?: boolean
parsedFrom?: string
[key: string]: any
}
export interface ProgressAwarePlayObject extends PlayObject {
meta: PlayMeta & {
initialTrackProgressPosition?: number
}
}
export type GroupedPlays = Map<string, ProgressAwarePlayObject[]>;
export type GroupedFixedPlays = Map<string, FixedSizeList<ProgressAwarePlayObject>>;
export interface TrackStringOptions {
include?: ('time' | 'artist' | 'track' | 'timeFromNow' | 'sourceId')[]
include?: ('time' | 'artist' | 'track' | 'timeFromNow' | 'trackId')[]
transformers?: {
artists?: (a: string[]) => string
track?: (t: string) => string
@@ -134,3 +184,54 @@ export interface RemoteIdentityParts {
proxy: string | undefined,
agent: string | undefined
}
export type LogLevel = "error" | "warn" | "info" | "verbose" | "debug";
export const logLevels = ['error', 'warn', 'info', 'verbose', 'debug'];
export interface LogConfig {
level?: string
file?: string | false
stream?: string
console?: string
}
export interface LogOptions {
/**
* Specify the minimum log level for all log outputs without their own level specified.
*
* Defaults to env `LOG_LEVEL` or `info` if not specified.
*
* @default 'info'
* */
level?: LogLevel
/**
* Specify the minimum log level to output to rotating files. If `false` no log files will be created.
* */
file?: LogLevel | false
/**
* Specify the minimum log level streamed to the UI
* */
stream?: LogLevel
/**
* Specify the minimum log level streamed to the console (or docker container)
* */
console?: LogLevel
}
export const asLogOptions = (obj: LogConfig = {}): obj is LogOptions => {
return Object.entries(obj).every(([key, val]) => {
if(key !== 'file') {
return val === undefined || logLevels.includes(val.toLocaleLowerCase());
}
return val === undefined || val === false || logLevels.includes(val.toLocaleLowerCase());
});
}
export interface LogInfo {
message: string
[MESSAGE]: string,
level: string
timestamp: string
labels?: string[]
transport?: string[]
}
@@ -2,12 +2,18 @@ import {SourceRetryOptions} from "./source/index.js";
import {RequestRetryOptions} from "./common.js";
import {SourceAIOConfig} from "./source/sources.js";
import {ClientAIOConfig} from "./client/clients.js";
import {WebhookConfig} from "./health/webhooks.js";
import {LogOptions} from "../Atomic.js";
export interface AIOConfig {
sourceDefaults?: SourceRetryOptions
clientDefaults?: RequestRetryOptions
sources?: SourceAIOConfig[]
clients?: ClientAIOConfig[]
webhooks?: WebhookConfig[]
logging?: LogOptions
}
export interface AIOClientConfig {
@@ -1,6 +1,7 @@
import {MalojaClientAIOConfig, MalojaClientConfig} from "./maloja.js";
import {LastfmClientAIOConfig, LastfmClientConfig} from "./lastfm.js";
import {ListenBrainzClientAIOConfig, ListenBrainzClientConfig} from "./listenbrainz.js";
export type ClientConfig = MalojaClientConfig | LastfmClientConfig;
export type ClientConfig = MalojaClientConfig | LastfmClientConfig | ListenBrainzClientConfig;
export type ClientAIOConfig = MalojaClientAIOConfig | LastfmClientAIOConfig;
export type ClientAIOConfig = MalojaClientAIOConfig | LastfmClientAIOConfig | ListenBrainzClientAIOConfig;
@@ -0,0 +1,40 @@
import {CommonClientConfig, CommonClientData} from "./index.js";
import {RequestRetryOptions} from "../common.js";
export interface ListenBrainzData extends RequestRetryOptions{
/**
* URL for the ListenBrainz server, if not using the default
*
* @examples ["https://api.listenbrainz.org/"]
* @default "https://api.listenbrainz.org/"
* */
url?: string
/**
* User token for the user to scrobble for
*
* @examples ["6794186bf-1157-4de6-80e5-uvb411f3ea2b"]
* */
token: string
/**
* Username of the user to scrobble for
* */
username: string
}
export interface ListenBrainzClientData extends ListenBrainzData, CommonClientData {}
export interface ListenBrainzClientConfig extends CommonClientConfig {
/**
* Should always be `client` when using Listenbrainz as a client
*
* @default client
* @examples ["client"]
* */
configureAs?: 'client' | 'source'
data: ListenBrainzClientData
}
export interface ListenBrainzClientAIOConfig extends ListenBrainzClientConfig {
type: 'listenbrainz'
}
@@ -24,3 +24,22 @@ export interface RequestRetryOptions {
* */
retryMultiplier?: number
}
export interface PollingOptions {
/**
* How long to wait before polling the source API for new tracks (in seconds)
*
* @default 30
* @examples [30]
* */
interval?: number
/**
* When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)
*
* @default 60
* @examples [60]
* */
maxInterval?: number
}
@@ -0,0 +1,93 @@
export interface WebhookPayload {
title?: string
message: string
priority: 'info' | 'warn' | 'error'
}
export interface PrioritiesConfig {
/**
* @examples [5]
* */
info: number
/**
* @examples [7]
* */
warn: number
/**
* @examples [10]
* */
error: number
}
export interface CommonWebhookConfig {
/**
* Webhook type. Valid values are:
*
* * gotify
* * ntfy
*
* @examples ["gotify"]
* */
type: 'gotify' | 'ntfy'
/**
* A friendly name used to identify webhook config in logs
* */
name?: string
}
export interface GotifyConfig extends CommonWebhookConfig {
/**
* The URL of the Gotify server. Same URL that would be used to reach the Gotify UI
*
* @examples ["http://192.168.0.100:8078"]
* */
url: string
/**
* The token created for this Application in Gotify
*
* @examples ["AQZI58fA.rfSZbm"]
* */
token: string
/**
* Priority of messages
*
* * Info -> 5
* * Warn -> 7
* * Error -> 10
* */
priorities?: PrioritiesConfig
}
export interface NtfyConfig extends CommonWebhookConfig {
/**
* The URL of the Ntfy server
*
* @examples ["http://192.168.0.100:8078"]
* */
url: string
/**
* The topic mutli-scrobbler should POST to
* */
topic: string
/**
* Required if topic is protected
* */
username?: string
/**
* Required if topic is protected
* */
password?: string
/**
* Priority of messages
*
* * Info -> 3
* * Warn -> 4
* * Error -> 5
* */
priorities?: PrioritiesConfig
}
export type WebhookConfig = GotifyConfig | NtfyConfig;
@@ -4,8 +4,8 @@ export interface SourceRetryOptions extends RequestRetryOptions {
/**
* default # of automatic polling restarts on error
*
* @default 0
* @examples [1]
* @default 5
* @examples [5]
* */
maxPollRetries?: number
}
@@ -17,6 +17,19 @@ export interface JellyData extends CommonSourceData {
* @examples [["MyServerName1"]]
* */
servers?: string | string[]
/**
* Additional options for jellyfin logging and tuning
* */
options?: {
/**
* Log raw Jellyfin webhook payload to debug
*
* @default false
* @examples [false]
* */
logPayload?: boolean
}
}
export interface JellySourceConfig extends CommonSourceConfig {
@@ -0,0 +1,41 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface JRiverData extends CommonSourceData, PollingOptions {
/**
* URL of the JRiver HTTP server to connect to
*
* multi-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:52199/MCWS/v1/`
*
* The URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.
*
* Parts => [default value]
*
* * Protocol => `http://`
* * Hostname => `localhost`
* * Port => `52199`
* * Path => `/MCWS/v1/`
*
*
* @examples ["http://localhost:52199/MCWS/v1/"]
* @default "http://localhost:52199/MCWS/v1/"
* */
url: string
/**
* If you have enabled authentication, the username you set
* */
username?: string
/**
* If you have enabled authentication, the password you set
* */
password?: string
}
export interface JRiverSourceConfig extends CommonSourceConfig {
data: JRiverData
}
export interface JRiverSourceAIOConfig extends JRiverSourceConfig {
type: 'jriver'
}
@@ -0,0 +1,42 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface KodiData extends CommonSourceData, PollingOptions {
/**
* URL of the Kodi HTTP server to connect to
*
* multi-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:8080/jsonrpc`
*
* The URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.
*
* Parts => [default value]
*
* * Protocol => `http://`
* * Hostname => `localhost`
* * Port => `8080`
* * Path => `/jsonrpc`
*
*
* @examples ["http://localhost:8080/jsonrpc"]
* @default "http://localhost:8080/jsonrpc"
* */
url: string
/**
* The username set for Remote Control via Web Sever
* */
username: string
/**
* The password set for Remote Control via Web Sever
* */
password: string
}
export interface KodiSourceConfig extends CommonSourceConfig {
data: KodiData
}
export interface KodiSourceAIOConfig extends KodiSourceConfig {
type: 'kodi'
}
@@ -0,0 +1,20 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {ListenBrainzData} from "../client/listenbrainz.js";
export interface ListenBrainzSourceData extends ListenBrainzData, CommonSourceData {
}
export interface ListenBrainzSourceConfig extends CommonSourceConfig {
/**
* When used in `listenbrainz.config` this tells multi-scrobbler whether to use this data to configure a source or client.
*
* @default source
* @examples ["source"]
* */
configureAs?: 'source'
data: ListenBrainzSourceData
}
export interface ListenBrainzSourceAIOConfig extends ListenBrainzSourceConfig {
type: 'listenbrainz'
}
@@ -0,0 +1,76 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface MopidyData extends CommonSourceData, PollingOptions {
/**
* URL of the Mopidy HTTP server to connect to
*
* You MUST have Mopidy-HTTP extension enabled: https://mopidy.com/ext/http
*
* multi-scrobbler connects to the WebSocket endpoint that ultimately looks like this => `ws://localhost:6680/mopidy/ws/`
*
* The URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.
*
* Parts => [default value]
*
* * Protocol => `ws://`
* * Hostname => `localhost`
* * Port => `6680`
* * Path => `/mopidy/ws/`
*
*
* @examples ["ws://localhost:6680/mopidy/ws/"]
* @default "ws://localhost:6680/mopidy/ws/"
* */
url?: string
/**
* Do not scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive
*
* EX: Don't scrobble tracks from soundcloud by adding 'soundcloud' to this list.
*
* List is ignored if uriWhitelist is used.
* */
uriBlacklist?: string[]
/**
* Only scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive
*
* EX: Only scrobble tracks from soundcloud by adding 'soundcloud' to this list.
*
* */
uriWhitelist?: string[]
/**
* Remove album data that matches any case-insensitive string from this list when scrobbling,
*
* For certain sources (Soundcloud) Mopidy does not have all track info (Album) and will instead use "Soundcloud" as the Album name. You can prevent multi-scrobbler from using this bad Album data by adding the fake name to this list. Multi-scrobbler will still scrobble the track, just without the bad data.
*
* @examples [["Soundcloud", "Mixcloud"]]
* @default ["Soundcloud"]
* */
albumBlacklist?: string[]
/**
* How long to wait before polling the source API for new tracks (in seconds)
*
* @default 10
* @examples [10]
* */
interval?: number
/**
* When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)
*
* @default 30
* @examples [30]
* */
maxInterval?: number
}
export interface MopidySourceConfig extends CommonSourceConfig {
data: MopidyData
}
export interface MopidySourceAIOConfig extends MopidySourceConfig {
type: 'mopidy'
}
@@ -0,0 +1,55 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
export const PLAYBACK_STATUS_PLAYING = 'Playing';
export const PLAYBACK_STATUS_PAUSED = 'Paused';
export const PLAYBACK_STATUS_STOPPED = 'Stopped';
export type PlaybackStatus = 'Playing' | 'Paused' | 'Stopped';
export const MPRIS_IFACE = 'org.mpris.MediaPlayer2.Player';
export const MPRIS_PATH = '/org/mpris/MediaPlayer2';
export const PROPERTIES_IFACE = 'org.freedesktop.DBus.Properties';
export interface MPRISMetadata {
trackid?: string
length?: number
artUrl?: string
album?: string
albumArtist?: string[]
artist?: string[]
title?: string
url?: string
}
export interface PlayerInfo {
name: string
status: PlaybackStatus
position?: number
metadata: MPRISMetadata
}
export interface MPRISData extends CommonSourceData {
/**
* DO NOT scrobble from any players that START WITH these values, case-insensitive
*
* @examples [["spotify","vlc"]]
* */
blacklist?: string | string[]
/**
* ONLY from any players that START WITH these values, case-insensitive
*
* If whitelist is present then blacklist is ignored
*
* @examples [["spotify","vlc"]]
* */
whitelist?: string | string[]
}
export interface MPRISSourceConfig extends CommonSourceConfig {
data: MPRISData
}
export interface MPRISSourceAIOConfig extends MPRISSourceConfig {
type: 'mpris'
}
@@ -5,7 +5,13 @@ import {DeezerSourceAIOConfig, DeezerSourceConfig} from "./deezer.js";
import {SubsonicSourceAIOConfig, SubSonicSourceConfig} from "./subsonic.js";
import {JellySourceAIOConfig, JellySourceConfig} from "./jellyfin.js";
import {LastFmSouceAIOConfig, LastfmSourceConfig} from "./lastfm.js";
import {YTMusicSourceAIOConfig, YTMusicSourceConfig} from "./ytmusic.js";
import {MPRISSourceAIOConfig, MPRISSourceConfig} from "./mpris.js";
import {MopidySourceAIOConfig, MopidySourceConfig} from "./mopidy.js";
import {ListenBrainzSourceAIOConfig, ListenBrainzSourceConfig} from "./listenbrainz.js";
import {JRiverSourceAIOConfig, JRiverSourceConfig} from "./jriver.js";
import {KodiSourceAIOConfig, KodiSourceConfig} from "./kodi.js";
export type SourceConfig = SpotifySourceConfig | PlexSourceConfig | TautulliSourceConfig | DeezerSourceConfig | SubSonicSourceConfig | JellySourceConfig | LastfmSourceConfig;
export type SourceConfig = SpotifySourceConfig | PlexSourceConfig | TautulliSourceConfig | DeezerSourceConfig | SubSonicSourceConfig | JellySourceConfig | LastfmSourceConfig | YTMusicSourceConfig | MPRISSourceConfig | MopidySourceConfig | ListenBrainzSourceConfig | JRiverSourceConfig | KodiSourceConfig;
export type SourceAIOConfig = SpotifySourceAIOConfig | PlexSourceAIOConfig | TautulliSourceAIOConfig | DeezerSourceAIOConfig | SubsonicSourceAIOConfig | JellySourceAIOConfig | LastFmSouceAIOConfig;
export type SourceAIOConfig = SpotifySourceAIOConfig | PlexSourceAIOConfig | TautulliSourceAIOConfig | DeezerSourceAIOConfig | SubsonicSourceAIOConfig | JellySourceAIOConfig | LastFmSouceAIOConfig | YTMusicSourceAIOConfig | MPRISSourceAIOConfig | MopidySourceAIOConfig | ListenBrainzSourceAIOConfig | JRiverSourceAIOConfig | KodiSourceAIOConfig;
@@ -1,6 +1,7 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface SpotifySourceData extends CommonSourceData {
export interface SpotifySourceData extends CommonSourceData, PollingOptions {
/**
* spotify client id
*
@@ -21,10 +22,17 @@ export interface SpotifySourceData extends CommonSourceData {
* */
redirectUri: string
/**
* optional, how long to wait before calling spotify for new tracks (in seconds)
* How long to wait before polling the source API for new tracks (in seconds)
*
* @default 60
* @examples [60]
* It is unlikely you should need to change this unless you scrobble many very short tracks often
*
* Reading:
* * https://developer.spotify.com/documentation/web-api/guides/rate-limits/
* * https://medium.com/mendix/limiting-your-amount-of-calls-in-mendix-most-of-the-time-rest-835dde55b10e
* * The rate limit is ~180 req/min
*
* @default 30
* @examples [30]
* */
interval?: number
}
@@ -1,6 +1,7 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface SubsonicData extends CommonSourceData {
export interface SubsonicData extends CommonSourceData, PollingOptions {
/**
* URL of the subsonic media server to query
*
@@ -20,6 +21,22 @@ export interface SubsonicData extends CommonSourceData {
* @examples ["MyPassword"]
* */
password: string
/**
* How long to wait before polling the source API for new tracks (in seconds)
*
* @default 10
* @examples [10]
* */
interval?: number
/**
* When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)
*
* @default 30
* @examples [30]
* */
maxInterval?: number
}
export interface SubSonicSourceConfig extends CommonSourceConfig {
data: SubsonicData
@@ -0,0 +1,26 @@
import {CommonSourceConfig, CommonSourceData} from "./index.js";
import {PollingOptions} from "../common.js";
export interface YTMusicData extends CommonSourceData, PollingOptions {
/**
* The cookie retrieved from the Request Headers of music.youtube.com after logging in.
*
* See https://github.com/nickp10/youtube-music-ts-api/blob/master/DOCUMENTATION.md#authenticate and https://ytmusicapi.readthedocs.io/en/latest/setup.html#copy-authentication-headers for how to retrieve this value.
*
* @examples ["VISITOR_INFO1_LIVE=jMp2xA1Xz2_PbVc; __Secure-3PAPISID=3AxsXpy0M/AkISpjek; ..."]
* */
cookie: string
/**
* If the 'X-Goog-AuthUser' header is present in the Request Headers for music.youtube.com it must also be included
*
* @example [0]
* */
authUser?: number
}
export interface YTMusicSourceConfig extends CommonSourceConfig {
data: YTMusicData
}
export interface YTMusicSourceAIOConfig extends YTMusicSourceConfig {
type: 'ytmusic'
}
+341
View File
@@ -0,0 +1,341 @@
import path from "path";
import {projectDir} from "./index.js";
import winston, {format, Logger} from "winston";
import {DuplexTransport} from "winston-duplex";
import {asLogOptions, LogConfig, LogInfo, LogLevel, LogOptions} from "./infrastructure/Atomic.js";
import process from "process";
import {fileOrDirectoryIsWriteable, truncateStringToLength} from "../utils.js";
import {ErrorWithCause, stackWithCauses} from "pony-cause";
import {NullTransport} from 'winston-null';
import 'winston-daily-rotate-file';
import dayjs from "dayjs";
import stringify from 'safe-stable-stringify';
import {SPLAT, LEVEL, MESSAGE} from 'triple-beam';
import {Symbol} from "typescript-json-schema";
const {combine, printf, timestamp, label, splat, errors} = format;
const {transports} = winston;
export let logPath = path.resolve(projectDir, `./logs`);
if (typeof process.env.CONFIG_DIR === 'string') {
logPath = path.resolve(process.env.CONFIG_DIR, './logs');
}
winston.loggers.add('noop', {transports: [new NullTransport()]});
export const getLogger = (config: LogConfig = {}, name = 'app'): Logger => {
if (!winston.loggers.has(name)) {
const errors: (Error | string)[] = [];
let options: LogOptions = {};
if (asLogOptions(config)) {
options = config;
} else {
errors.push(`Logging levels were not valid. Must be one of: 'error', 'warn', 'info', 'verbose', 'debug' -- 'file' may be false.`);
}
const {level: configLevel} = options;
const defaultLevel = process.env.LOG_LEVEL || 'info';
const {
level = configLevel || defaultLevel,
file = configLevel || defaultLevel,
stream = configLevel || 'debug',
console = configLevel || 'debug'
} = options;
const consoleTransport = new transports.Console({level: console});
const myTransports = [
consoleTransport,
new DuplexTransport({
stream: {
transform(chunk, e, cb) {
cb(null, chunk);
},
objectMode: true,
},
name: 'duplex',
handleExceptions: true,
handleRejections: true,
level: stream,
dump: false,
}),
];
if (file !== false) {
const rotateTransport = new winston.transports.DailyRotateFile({
dirname: logPath,
createSymlink: true,
symlinkName: 'scrobble-current.log',
filename: 'scrobble-%DATE%.log',
datePattern: 'YYYY-MM-DD',
maxSize: '5m',
level: file
});
try {
fileOrDirectoryIsWriteable(logPath);
// @ts-ignore
myTransports.push(rotateTransport);
} catch (e: any) {
let msg = 'WILL NOT write logs to rotating file due to an error while trying to access the specified logging directory';
errors.push(new ErrorWithCause<Error>(msg, {cause: e}));
}
}
const loggerOptions: winston.LoggerOptions = {
level: level,
format: labelledFormat(),
transports: myTransports,
};
winston.loggers.add(name, loggerOptions);
const logger = winston.loggers.get(name);
if (errors.length > 0) {
for (const e of errors) {
logger.error(e);
}
}
return logger;
}
return winston.loggers.get(name);
}
const breakSymbol = '<br />';
export const formatLogToHtml = (chunk: any) => {
const line = chunk.toString().replace('\n', breakSymbol)
.replace(/(debug)\s/gi, '<span class="debug blue">$1 </span>')
.replace(/(warn)\s/gi, '<span class="warn yellow">$1 </span>')
.replace(/(info)\s/gi, '<span class="info green">$1 </span>')
.replace(/(verbose)\s/gi, '<span class="verbose purple">$1 </span>')
.replace(/(error)\s/gi, '<span class="error red">$1 </span>')
.trim();
if(line.slice(-6) !== breakSymbol) {
return `${line}${breakSymbol}`;
}
return line;
}
const levelSymbol = Symbol.for('level');
const s = splat();
//const errorsFormat = errors({stack: true});
const CWD = process.cwd();
const causeKeys = ['name', 'cause']
export const defaultFormat = (defaultLabel = 'App') => printf(({
label,
[levelSymbol]: levelSym,
level,
message,
labels = [defaultLabel],
leaf,
timestamp,
durationMs,
[SPLAT]: splatObj,
stack,
...rest
}) => {
const keys = Object.keys(rest);
let stringifyValue = keys.length > 0 && !keys.every(x => causeKeys.some(y => y == x)) ? stringify(rest) : '';
let msg = message;
let stackMsg = '';
if (stack !== undefined) {
const stackArr = stack.split('\n');
const stackTop = stackArr[0];
const cleanedStack = stackArr
.slice(1) // don't need actual error message since we are showing it as msg
.map((x: string) => x.replace(CWD, 'CWD')) // replace file location up to cwd for user privacy
.join('\n'); // rejoin with newline to preserve formatting
stackMsg = `\n${cleanedStack}`;
if (msg === undefined || msg === null || typeof message === 'object') {
msg = stackTop;
} else {
stackMsg = `\n${stackTop}${stackMsg}`
}
}
let nodes = Array.isArray(labels) ? labels : [labels];
if (leaf !== null && leaf !== undefined && !nodes.includes(leaf)) {
nodes.push(leaf);
}
const labelContent = `${nodes.map((x: string) => `[${x}]`).join(' ')}`;
return `${timestamp} ${level.padEnd(8)}: ${labelContent} ${msg}${stringifyValue !== '' ? ` ${stringifyValue}` : ''}${stackMsg}`;
});
export const labelledFormat = (labelName = 'App') => {
const l = label({label: labelName, message: false});
return combine(
timestamp(
{
format: () => dayjs().local().format(),
}
),
l,
s,
errorAwareFormat,
defaultFormat(labelName),
);
}
export const logLevels = {
error: 0,
warn: 1,
info: 2,
http: 3,
verbose: 4,
debug: 5,
trace: 5,
silly: 6
};
export const LOG_LEVEL_REGEX: RegExp = /\s*(debug|warn|info|error|verbose)\s*:/i
export const isLogLineMinLevel = (log: string | LogInfo, minLevelText: LogLevel): boolean => {
// @ts-ignore
const minLevel = logLevels[minLevelText];
let level: number;
if(typeof log === 'string') {
const lineLevelMatch = log.match(LOG_LEVEL_REGEX)
if (lineLevelMatch === null) {
return false;
}
// @ts-ignore
level = logLevels[lineLevelMatch[1]];
} else {
const lineLevelMatch = log.level;
// @ts-ignore
level = logLevels[lineLevelMatch];
}
return level <= minLevel;
}
const isProbablyError = (val: any, explicitErrorName?: string) => {
if(typeof val !== 'object' || val === null) {
return false;
}
const {name, stack} = val;
if(explicitErrorName !== undefined) {
if(name !== undefined && name.toLowerCase().includes(explicitErrorName)) {
return true;
}
if(stack !== undefined && stack.trim().toLowerCase().indexOf(explicitErrorName.toLowerCase()) === 0) {
return true;
}
return false;
} else if(stack !== undefined) {
return true;
} else if(name !== undefined && name.toLowerCase().includes('error')) {
return true;
}
return false;
}
const errorAwareFormat = {
transform: (einfo: any, {stack = true}: any = {}) => {
// because winston logger.child() re-assigns its input to an object ALWAYS the object we recieve here will never actually be of type Error
const includeStack = stack && (!isProbablyError(einfo, 'simpleerror') && !isProbablyError(einfo.message, 'simpleerror'));
if (!isProbablyError(einfo.message) && !isProbablyError(einfo)) {
return einfo;
}
let info: any = {};
if (isProbablyError(einfo)) {
const tinfo = transformError(einfo);
info = Object.assign({}, tinfo, {
// @ts-ignore
level: einfo.level,
// @ts-ignore
[LEVEL]: einfo[LEVEL] || einfo.level,
message: tinfo.message,
// @ts-ignore
[MESSAGE]: tinfo[MESSAGE] || tinfo.message
});
if(includeStack) {
// so we have to create a dummy error and re-assign all error properties from our info object to it so we can get a proper stack trace
const dummyErr = new ErrorWithCause('');
const names = Object.getOwnPropertyNames(tinfo);
for(const k of names) {
if(dummyErr.hasOwnProperty(k) || k === 'cause') {
// @ts-ignore
dummyErr[k] = tinfo[k];
}
}
// @ts-ignore
info.stack = stackWithCauses(dummyErr);
}
} else {
const err = transformError(einfo.message);
info = Object.assign({}, einfo, err);
// @ts-ignore
info.message = err.message;
// @ts-ignore
info[MESSAGE] = err.message;
if(includeStack) {
const dummyErr = new ErrorWithCause('');
// Error properties are not enumerable
// https://stackoverflow.com/a/18278145/1469797
const names = Object.getOwnPropertyNames(err);
for(const k of names) {
if(dummyErr.hasOwnProperty(k) || k === 'cause') {
// @ts-ignore
dummyErr[k] = err[k];
}
}
// @ts-ignore
info.stack = stackWithCauses(dummyErr);
}
}
// remove redundant message from stack and make stack causes easier to read
if(info.stack !== undefined) {
let cleanedStack = info.stack.replace(info.message, '');
cleanedStack = `${cleanedStack}`;
cleanedStack = cleanedStack.replaceAll('caused by:', '\ncaused by:');
info.stack = cleanedStack;
}
return info;
}
}
export const transformError = (err: Error): any => _transformError(err, new Set());
const _transformError = (err: Error, seen: Set<Error>) => {
if (!err || !isProbablyError(err)) {
return '';
}
if (seen.has(err)) {
return err;
}
try {
// @ts-ignore
let mOpts = err.matchOptions ?? matchOptions;
// @ts-ignore
const cause = err.cause as unknown;
if (cause !== undefined && cause instanceof Error) {
// @ts-ignore
err.cause = _transformError(cause, seen, mOpts);
}
return err;
} catch (e: any) {
// oops :(
// we're gonna swallow silently instead of reporting to avoid any infinite nesting and hopefully the original error looks funny enough to provide clues as to what to fix here
return err;
}
}
+116
View File
@@ -6,6 +6,9 @@
{
"$ref": "#/definitions/LastfmClientAIOConfig"
},
{
"$ref": "#/definitions/ListenBrainzClientAIOConfig"
},
{
"$ref": "#/definitions/MalojaClientAIOConfig"
}
@@ -159,6 +162,119 @@
],
"type": "object"
},
"ListenBrainzClientAIOConfig": {
"properties": {
"configureAs": {
"default": "client",
"description": "Should always be `client` when using Listenbrainz as a client",
"enum": [
"client",
"source"
],
"examples": [
"client"
],
"type": "string"
},
"data": {
"$ref": "#/definitions/ListenBrainzClientData",
"description": "Specific data required to configure this client"
},
"name": {
"description": "Unique identifier for this client. Used with sources to restrict where scrobbles are sent.",
"examples": [
"MyConfig"
],
"type": "string"
},
"type": {
"enum": [
"listenbrainz"
],
"type": "string"
}
},
"required": [
"data",
"name",
"type"
],
"type": "object"
},
"ListenBrainzClientData": {
"properties": {
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"properties": {
"checkExistingScrobbles": {
"default": true,
"description": "Check client for an existing scrobble at the same recorded time as the \"new\" track to be scrobbled. If an existing scrobble is found this track is not track scrobbled.",
"examples": [
true
],
"type": "boolean"
},
"refreshEnabled": {
"default": true,
"description": "Try to get fresh scrobble history from client when tracks to be scrobbled are newer than the last scrobble found in client history",
"examples": [
true
],
"type": "boolean"
},
"verbose": {
"description": "Options used for increasing verbosity of logging in MS (used for debugging)",
"properties": {
"match": {
"$ref": "#/definitions/MatchLoggingOptions"
}
},
"type": "object"
}
},
"type": "object"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"token": {
"description": "User token for the user to scrobble for",
"examples": [
"6794186bf-1157-4de6-80e5-uvb411f3ea2b"
],
"type": "string"
},
"url": {
"default": "https://api.listenbrainz.org/",
"description": "URL for the ListenBrainz server, if not using the default",
"examples": [
"https://api.listenbrainz.org/"
],
"type": "string"
},
"username": {
"description": "Username of the user to scrobble for",
"type": "string"
}
},
"required": [
"token",
"username"
],
"type": "object"
},
"MalojaClientAIOConfig": {
"properties": {
"data": {
+701 -16
View File
@@ -26,10 +26,10 @@
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -103,13 +103,29 @@
],
"type": "object"
},
"JellyData": {
"JRiverData": {
"properties": {
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -124,6 +140,103 @@
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"password": {
"description": "If you have enabled authentication, the password you set",
"type": "string"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"url": {
"default": "http://localhost:52199/MCWS/v1/",
"description": "URL of the JRiver HTTP server to connect to\n\nmulti-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:52199/MCWS/v1/`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `http://`\n* Hostname => `localhost`\n* Port => `52199`\n* Path => `/MCWS/v1/`",
"examples": [
"http://localhost:52199/MCWS/v1/"
],
"type": "string"
},
"username": {
"description": "If you have enabled authentication, the username you set",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"JRiverSourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/JRiverData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"jriver"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
"JellyData": {
"properties": {
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"description": "Additional options for jellyfin logging and tuning",
"properties": {
"logPayload": {
"default": false,
"description": "Log raw Jellyfin webhook payload to debug",
"examples": [
false
],
"type": "boolean"
}
},
"type": "object"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
@@ -209,6 +322,110 @@
],
"type": "object"
},
"KodiData": {
"properties": {
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"password": {
"description": "The password set for Remote Control via Web Sever",
"type": "string"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"url": {
"default": "http://localhost:8080/jsonrpc",
"description": "URL of the Kodi HTTP server to connect to\n\nmulti-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:8080/jsonrpc`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `http://`\n* Hostname => `localhost`\n* Port => `8080`\n* Path => `/jsonrpc`",
"examples": [
"http://localhost:8080/jsonrpc"
],
"type": "string"
},
"username": {
"description": "The username set for Remote Control via Web Sever",
"type": "string"
}
},
"required": [
"password",
"url",
"username"
],
"type": "object"
},
"KodiSourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/KodiData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"kodi"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
"LastFmSouceAIOConfig": {
"properties": {
"clients": {
@@ -265,10 +482,10 @@
"type": "string"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -317,6 +534,335 @@
],
"type": "object"
},
"ListenBrainzSourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"configureAs": {
"default": "source",
"description": "When used in `listenbrainz.config` this tells multi-scrobbler whether to use this data to configure a source or client.",
"enum": [
"source"
],
"examples": [
"source"
],
"type": "string"
},
"data": {
"$ref": "#/definitions/ListenBrainzSourceData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"listenbrainz"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
"ListenBrainzSourceData": {
"properties": {
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"token": {
"description": "User token for the user to scrobble for",
"examples": [
"6794186bf-1157-4de6-80e5-uvb411f3ea2b"
],
"type": "string"
},
"url": {
"default": "https://api.listenbrainz.org/",
"description": "URL for the ListenBrainz server, if not using the default",
"examples": [
"https://api.listenbrainz.org/"
],
"type": "string"
},
"username": {
"description": "Username of the user to scrobble for",
"type": "string"
}
},
"required": [
"token",
"username"
],
"type": "object"
},
"MPRISData": {
"properties": {
"blacklist": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "string"
}
],
"description": "DO NOT scrobble from any players that START WITH these values, case-insensitive",
"examples": [
[
"spotify",
"vlc"
]
]
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"whitelist": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "string"
}
],
"description": "ONLY from any players that START WITH these values, case-insensitive\n\nIf whitelist is present then blacklist is ignored",
"examples": [
[
"spotify",
"vlc"
]
]
}
},
"type": "object"
},
"MPRISSourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/MPRISData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"mpris"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
"MopidyData": {
"properties": {
"albumBlacklist": {
"default": [
"Soundcloud"
],
"description": "Remove album data that matches any case-insensitive string from this list when scrobbling,\n\nFor certain sources (Soundcloud) Mopidy does not have all track info (Album) and will instead use \"Soundcloud\" as the Album name. You can prevent multi-scrobbler from using this bad Album data by adding the fake name to this list. Multi-scrobbler will still scrobble the track, just without the bad data.",
"examples": [
[
"Soundcloud",
"Mixcloud"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"interval": {
"default": 10,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
10
],
"type": "number"
},
"maxInterval": {
"default": 30,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"uriBlacklist": {
"description": "Do not scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive\n\nEX: Don't scrobble tracks from soundcloud by adding 'soundcloud' to this list.\n\nList is ignored if uriWhitelist is used.",
"items": {
"type": "string"
},
"type": "array"
},
"uriWhitelist": {
"description": "Only scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive\n\nEX: Only scrobble tracks from soundcloud by adding 'soundcloud' to this list.",
"items": {
"type": "string"
},
"type": "array"
},
"url": {
"default": "ws://localhost:6680/mopidy/ws/",
"description": "URL of the Mopidy HTTP server to connect to\n\nYou MUST have Mopidy-HTTP extension enabled: https://mopidy.com/ext/http\n\nmulti-scrobbler connects to the WebSocket endpoint that ultimately looks like this => `ws://localhost:6680/mopidy/ws/`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `ws://`\n* Hostname => `localhost`\n* Port => `6680`\n* Path => `/mopidy/ws/`",
"examples": [
"ws://localhost:6680/mopidy/ws/"
],
"type": "string"
}
},
"type": "object"
},
"MopidySourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/MopidyData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"mopidy"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
},
"PlexSourceAIOConfig": {
"properties": {
"clients": {
@@ -375,10 +921,10 @@
]
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -468,16 +1014,34 @@
},
{
"$ref": "#/definitions/LastFmSouceAIOConfig"
},
{
"$ref": "#/definitions/YTMusicSourceAIOConfig"
},
{
"$ref": "#/definitions/MPRISSourceAIOConfig"
},
{
"$ref": "#/definitions/MopidySourceAIOConfig"
},
{
"$ref": "#/definitions/ListenBrainzSourceAIOConfig"
},
{
"$ref": "#/definitions/JRiverSourceAIOConfig"
},
{
"$ref": "#/definitions/KodiSourceAIOConfig"
}
]
},
"SourceRetryOptions": {
"properties": {
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -552,18 +1116,26 @@
"type": "string"
},
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)\n\nIt is unlikely you should need to change this unless you scrobble many very short tracks often\n\nReading:\n* https://developer.spotify.com/documentation/web-api/guides/rate-limits/\n* https://medium.com/mendix/limiting-your-amount-of-calls-in-mendix-most-of-the-time-rest-835dde55b10e\n * The rate limit is ~180 req/min",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "optional, how long to wait before calling spotify for new tracks (in seconds)",
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -604,11 +1176,27 @@
},
"SubsonicData": {
"properties": {
"interval": {
"default": 10,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
10
],
"type": "number"
},
"maxInterval": {
"default": 30,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -726,6 +1314,103 @@
"type"
],
"type": "object"
},
"YTMusicData": {
"properties": {
"authUser": {
"description": "If the 'X-Goog-AuthUser' header is present in the Request Headers for music.youtube.com it must also be included",
"type": "number"
},
"cookie": {
"description": "The cookie retrieved from the Request Headers of music.youtube.com after logging in.\n\nSee https://github.com/nickp10/youtube-music-ts-api/blob/master/DOCUMENTATION.md#authenticate and https://ytmusicapi.readthedocs.io/en/latest/setup.html#copy-authentication-headers for how to retrieve this value.",
"examples": [
"VISITOR_INFO1_LIVE=jMp2xA1Xz2_PbVc; __Secure-3PAPISID=3AxsXpy0M/AkISpjek; ..."
],
"type": "string"
},
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
}
},
"required": [
"cookie"
],
"type": "object"
},
"YTMusicSourceAIOConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/YTMusicData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
},
"type": {
"enum": [
"ytmusic"
],
"type": "string"
}
},
"required": [
"data",
"type"
],
"type": "object"
}
},
"properties": {
File diff suppressed because it is too large Load Diff
+109
View File
@@ -4,6 +4,9 @@
{
"$ref": "#/definitions/LastfmClientConfig"
},
{
"$ref": "#/definitions/ListenBrainzClientConfig"
},
{
"$ref": "#/definitions/MalojaClientConfig"
}
@@ -150,6 +153,112 @@
],
"type": "object"
},
"ListenBrainzClientConfig": {
"properties": {
"configureAs": {
"default": "client",
"description": "Should always be `client` when using Listenbrainz as a client",
"enum": [
"client",
"source"
],
"examples": [
"client"
],
"type": "string"
},
"data": {
"$ref": "#/definitions/ListenBrainzClientData",
"description": "Specific data required to configure this client"
},
"name": {
"description": "Unique identifier for this client. Used with sources to restrict where scrobbles are sent.",
"examples": [
"MyConfig"
],
"type": "string"
}
},
"required": [
"data",
"name"
],
"type": "object"
},
"ListenBrainzClientData": {
"properties": {
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"properties": {
"checkExistingScrobbles": {
"default": true,
"description": "Check client for an existing scrobble at the same recorded time as the \"new\" track to be scrobbled. If an existing scrobble is found this track is not track scrobbled.",
"examples": [
true
],
"type": "boolean"
},
"refreshEnabled": {
"default": true,
"description": "Try to get fresh scrobble history from client when tracks to be scrobbled are newer than the last scrobble found in client history",
"examples": [
true
],
"type": "boolean"
},
"verbose": {
"description": "Options used for increasing verbosity of logging in MS (used for debugging)",
"properties": {
"match": {
"$ref": "#/definitions/MatchLoggingOptions"
}
},
"type": "object"
}
},
"type": "object"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"token": {
"description": "User token for the user to scrobble for",
"examples": [
"6794186bf-1157-4de6-80e5-uvb411f3ea2b"
],
"type": "string"
},
"url": {
"default": "https://api.listenbrainz.org/",
"description": "URL for the ListenBrainz server, if not using the default",
"examples": [
"https://api.listenbrainz.org/"
],
"type": "string"
},
"username": {
"description": "Username of the user to scrobble for",
"type": "string"
}
},
"required": [
"token",
"username"
],
"type": "object"
},
"MalojaClientConfig": {
"properties": {
"data": {
+657 -14
View File
@@ -21,6 +21,24 @@
},
{
"$ref": "#/definitions/LastfmSourceConfig"
},
{
"$ref": "#/definitions/YTMusicSourceConfig"
},
{
"$ref": "#/definitions/MPRISSourceConfig"
},
{
"$ref": "#/definitions/MopidySourceConfig"
},
{
"$ref": "#/definitions/ListenBrainzSourceConfig"
},
{
"$ref": "#/definitions/JRiverSourceConfig"
},
{
"$ref": "#/definitions/KodiSourceConfig"
}
],
"definitions": {
@@ -49,10 +67,10 @@
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -119,13 +137,29 @@
],
"type": "object"
},
"JellyData": {
"JRiverData": {
"properties": {
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -140,6 +174,96 @@
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"password": {
"description": "If you have enabled authentication, the password you set",
"type": "string"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"url": {
"default": "http://localhost:52199/MCWS/v1/",
"description": "URL of the JRiver HTTP server to connect to\n\nmulti-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:52199/MCWS/v1/`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `http://`\n* Hostname => `localhost`\n* Port => `52199`\n* Path => `/MCWS/v1/`",
"examples": [
"http://localhost:52199/MCWS/v1/"
],
"type": "string"
},
"username": {
"description": "If you have enabled authentication, the username you set",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"JRiverSourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/JRiverData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
},
"JellyData": {
"properties": {
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"description": "Additional options for jellyfin logging and tuning",
"properties": {
"logPayload": {
"default": false,
"description": "Log raw Jellyfin webhook payload to debug",
"examples": [
false
],
"type": "boolean"
}
},
"type": "object"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
@@ -218,6 +342,103 @@
],
"type": "object"
},
"KodiData": {
"properties": {
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"password": {
"description": "The password set for Remote Control via Web Sever",
"type": "string"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"url": {
"default": "http://localhost:8080/jsonrpc",
"description": "URL of the Kodi HTTP server to connect to\n\nmulti-scrobbler connects to the Web Service Interface endpoint that ultimately looks like this => `http://yourDomain:8080/jsonrpc`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `http://`\n* Hostname => `localhost`\n* Port => `8080`\n* Path => `/jsonrpc`",
"examples": [
"http://localhost:8080/jsonrpc"
],
"type": "string"
},
"username": {
"description": "The username set for Remote Control via Web Sever",
"type": "string"
}
},
"required": [
"password",
"url",
"username"
],
"type": "object"
},
"KodiSourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/KodiData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
},
"LastFmSourceData": {
"properties": {
"apiKey": {
@@ -228,10 +449,10 @@
"type": "string"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -319,6 +540,314 @@
],
"type": "object"
},
"ListenBrainzSourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"configureAs": {
"default": "source",
"description": "When used in `listenbrainz.config` this tells multi-scrobbler whether to use this data to configure a source or client.",
"enum": [
"source"
],
"examples": [
"source"
],
"type": "string"
},
"data": {
"$ref": "#/definitions/ListenBrainzSourceData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
},
"ListenBrainzSourceData": {
"properties": {
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"token": {
"description": "User token for the user to scrobble for",
"examples": [
"6794186bf-1157-4de6-80e5-uvb411f3ea2b"
],
"type": "string"
},
"url": {
"default": "https://api.listenbrainz.org/",
"description": "URL for the ListenBrainz server, if not using the default",
"examples": [
"https://api.listenbrainz.org/"
],
"type": "string"
},
"username": {
"description": "Username of the user to scrobble for",
"type": "string"
}
},
"required": [
"token",
"username"
],
"type": "object"
},
"MPRISData": {
"properties": {
"blacklist": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "string"
}
],
"description": "DO NOT scrobble from any players that START WITH these values, case-insensitive",
"examples": [
[
"spotify",
"vlc"
]
]
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"whitelist": {
"anyOf": [
{
"items": {
"type": "string"
},
"type": "array"
},
{
"type": "string"
}
],
"description": "ONLY from any players that START WITH these values, case-insensitive\n\nIf whitelist is present then blacklist is ignored",
"examples": [
[
"spotify",
"vlc"
]
]
}
},
"type": "object"
},
"MPRISSourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/MPRISData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
},
"MopidyData": {
"properties": {
"albumBlacklist": {
"default": [
"Soundcloud"
],
"description": "Remove album data that matches any case-insensitive string from this list when scrobbling,\n\nFor certain sources (Soundcloud) Mopidy does not have all track info (Album) and will instead use \"Soundcloud\" as the Album name. You can prevent multi-scrobbler from using this bad Album data by adding the fake name to this list. Multi-scrobbler will still scrobble the track, just without the bad data.",
"examples": [
[
"Soundcloud",
"Mixcloud"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"interval": {
"default": 10,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
10
],
"type": "number"
},
"maxInterval": {
"default": 30,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
},
"uriBlacklist": {
"description": "Do not scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive\n\nEX: Don't scrobble tracks from soundcloud by adding 'soundcloud' to this list.\n\nList is ignored if uriWhitelist is used.",
"items": {
"type": "string"
},
"type": "array"
},
"uriWhitelist": {
"description": "Only scrobble tracks whose URI STARTS WITH any of these strings, case-insensitive\n\nEX: Only scrobble tracks from soundcloud by adding 'soundcloud' to this list.",
"items": {
"type": "string"
},
"type": "array"
},
"url": {
"default": "ws://localhost:6680/mopidy/ws/",
"description": "URL of the Mopidy HTTP server to connect to\n\nYou MUST have Mopidy-HTTP extension enabled: https://mopidy.com/ext/http\n\nmulti-scrobbler connects to the WebSocket endpoint that ultimately looks like this => `ws://localhost:6680/mopidy/ws/`\n\nThe URL you provide here will have all parts not explicitly defined filled in for you so if these are not the default you must define them.\n\nParts => [default value]\n\n* Protocol => `ws://`\n* Hostname => `localhost`\n* Port => `6680`\n* Path => `/mopidy/ws/`",
"examples": [
"ws://localhost:6680/mopidy/ws/"
],
"type": "string"
}
},
"type": "object"
},
"MopidySourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/MopidyData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
},
"PlexSourceConfig": {
"properties": {
"clients": {
@@ -370,10 +899,10 @@
]
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -486,18 +1015,26 @@
"type": "string"
},
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)\n\nIt is unlikely you should need to change this unless you scrobble many very short tracks often\n\nReading:\n* https://developer.spotify.com/documentation/web-api/guides/rate-limits/\n* https://medium.com/mendix/limiting-your-amount-of-calls-in-mendix-most-of-the-time-rest-835dde55b10e\n * The rate limit is ~180 req/min",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "optional, how long to wait before calling spotify for new tracks (in seconds)",
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -566,11 +1103,27 @@
},
"SubsonicData": {
"properties": {
"interval": {
"default": 10,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
10
],
"type": "number"
},
"maxInterval": {
"default": 30,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxPollRetries": {
"default": 0,
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
1
5
],
"type": "number"
},
@@ -646,6 +1199,96 @@
"data"
],
"type": "object"
},
"YTMusicData": {
"properties": {
"authUser": {
"description": "If the 'X-Goog-AuthUser' header is present in the Request Headers for music.youtube.com it must also be included",
"type": "number"
},
"cookie": {
"description": "The cookie retrieved from the Request Headers of music.youtube.com after logging in.\n\nSee https://github.com/nickp10/youtube-music-ts-api/blob/master/DOCUMENTATION.md#authenticate and https://ytmusicapi.readthedocs.io/en/latest/setup.html#copy-authentication-headers for how to retrieve this value.",
"examples": [
"VISITOR_INFO1_LIVE=jMp2xA1Xz2_PbVc; __Secure-3PAPISID=3AxsXpy0M/AkISpjek; ..."
],
"type": "string"
},
"interval": {
"default": 30,
"description": "How long to wait before polling the source API for new tracks (in seconds)",
"examples": [
30
],
"type": "number"
},
"maxInterval": {
"default": 60,
"description": "When there has been no new activity from the Source API multi-scrobbler will gradually increase the wait time between polling up to this value (in seconds)",
"examples": [
60
],
"type": "number"
},
"maxPollRetries": {
"default": 5,
"description": "default # of automatic polling restarts on error",
"examples": [
5
],
"type": "number"
},
"maxRequestRetries": {
"default": 1,
"description": "default # of http request retries a source can make before error is thrown",
"examples": [
1
],
"type": "number"
},
"options": {
"$ref": "#/definitions/Record<string,any>"
},
"retryMultiplier": {
"default": 1.5,
"description": "default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying)",
"examples": [
1.5
],
"type": "number"
}
},
"required": [
"cookie"
],
"type": "object"
},
"YTMusicSourceConfig": {
"properties": {
"clients": {
"description": "Restrict scrobbling tracks played from this source to Clients with names from this list. If list is empty is not present Source scrobbles to all configured Clients.",
"examples": [
[
"MyMalojaConfigName",
"MyLastFMConfigName"
]
],
"items": {
"type": "string"
},
"type": "array"
},
"data": {
"$ref": "#/definitions/YTMusicData"
},
"name": {
"description": "Unique identifier for this source.",
"type": "string"
}
},
"required": [
"data"
],
"type": "object"
}
}
}
+111 -98
View File
@@ -1,7 +1,7 @@
import {addAsync, Router} from '@awaitjs/express';
import express from 'express';
import bodyParser from 'body-parser';
import winston from 'winston';
import {Logger} from 'winston';
import dayjs from 'dayjs';
import utc from 'dayjs/plugin/utc.js';
import isBetween from 'dayjs/plugin/isBetween.js';
@@ -9,36 +9,36 @@ import relativeTime from 'dayjs/plugin/relativeTime.js';
import duration from 'dayjs/plugin/duration.js';
import passport from 'passport';
import session from 'express-session';
import {Writable} from 'stream';
import 'winston-daily-rotate-file';
import {
buildTrackString,
capitalize, createLabelledLogger,
labelledFormat,
longestString,
readJson, sleep,
capitalize,
longestString, mergeArr,
readJson,
remoteHostIdentifiers,
sleep,
truncateStringToLength
} from "./utils.js";
import ScrobbleSources from "./sources/ScrobbleSources.js";
import {makeClientCheckMiddle, makeSourceCheckMiddle} from "./server/middleware.js";
import TautulliSource from "./sources/TautulliSource.js";
import PlexSource, {plexRequestMiddle} from "./sources/PlexSource.js";
import JellyfinSource from "./sources/JellyfinSource.js";
import { Server } from "socket.io";
import {Server} from "socket.io";
import * as path from "path";
import {projectDir} from "./common/index.js";
import LastfmApiClient from "./apis/LastfmApiClient.js";
import LastfmSource from "./sources/LastfmSource.js";
import LastfmScrobbler from "./clients/LastfmScrobbler.js";
import ScrobbleClients from "./clients/ScrobbleClients.js";
import DeezerSource from "./sources/DeezerSource.js";
import AbstractSource from "./sources/AbstractSource.js";
import {PlayObject, TrackStringOptions} from "./common/infrastructure/Atomic.js";
import {LogInfo, LogLevel, PlayObject, TrackStringOptions} from "./common/infrastructure/Atomic.js";
import SpotifySource from "./sources/SpotifySource.js";
import {JellyfinNotifier} from "./sources/ingressNotifiers/JellyfinNotifier.js";
import {PlexNotifier} from "./sources/ingressNotifiers/PlexNotifier.js";
import {TautulliNotifier} from "./sources/ingressNotifiers/TautulliNotifier.js";
import {AIOConfig} from "./common/infrastructure/config/aioConfig.js";
import createRoot from "./ioc.js";
import {formatLogToHtml, getLogger, isLogLineMinLevel} from "./common/logging.js";
import {MESSAGE} from "triple-beam";
dayjs.extend(utc)
@@ -63,92 +63,73 @@ app.use(session({secret: 'keyboard cat', resave: false, saveUninitialized: false
app.use(passport.initialize());
app.use(passport.session());
const {transports} = winston;
let output: LogInfo[] = []
let output: any = []
const stream = new Writable()
stream._write = (chunk, encoding, next) => {
let formatString = chunk.toString().replace('\n', '<br />')
.replace(/(debug)\s/gi, '<span class="debug text-pink-400">$1 </span>')
.replace(/(warn)\s/gi, '<span class="warn text-blue-400">$1 </span>')
.replace(/(info)\s/gi, '<span class="info text-yellow-500">$1 </span>')
.replace(/(error)\s/gi, '<span class="error text-red-400">$1 </span>')
output.unshift(formatString);
output = output.slice(0, 101);
io.emit('log', formatString);
next();
}
const streamTransport = new winston.transports.Stream({
stream,
})
const logConfig = {
level: process.env.LOG_LEVEL || 'info',
sort: 'descending',
limit: 50,
}
const availableLevels = ['info', 'debug'];
let logPath = path.resolve(projectDir, `./logs`);
if(typeof process.env.CONFIG_DIR === 'string') {
logPath = path.resolve(process.env.CONFIG_DIR, './logs');
}
const localUrl = `http://localhost:${port}`;
const rotateTransport = new winston.transports.DailyRotateFile({
dirname: logPath,
createSymlink: true,
symlinkName: 'scrobble-current.log',
filename: 'scrobble-%DATE%.log',
datePattern: 'YYYY-MM-DD',
maxSize: '5m'
const initLogger = getLogger({}, 'init');
initLogger.stream().on('log', (log: LogInfo) => {
output.unshift(log);
output = output.slice(0, 301);
io.emit('log', formatLogToHtml(log[MESSAGE]));
});
const consoleTransport = new transports.Console();
const myTransports = [
consoleTransport,
streamTransport,
];
if (typeof logPath === 'string') {
// @ts-ignore
myTransports.push(rotateTransport);
}
const loggerOptions: winston.LoggerOptions = {
level: logConfig.level,
format: labelledFormat(),
transports: myTransports,
};
winston.loggers.add('default', loggerOptions);
const logger = winston.loggers.get('default');
let logger: Logger;
const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
try {
// try to read a configuration file
let appConfigFail = false;
let config = {};
try {
config = await readJson(`${configDir}/config.json`, {throwOnNotFound: false});
} catch (e) {
appConfigFail = true;
}
const {
webhooks = [],
logging = {},
} = (config || {}) as AIOConfig;
const logConfig: {level: LogLevel, sort: string, limit: number} = {
level: logging.level || (process.env.LOG_LEVEL || 'info') as LogLevel,
sort: 'descending',
limit: 50,
}
logger = getLogger(logging, 'app');
logger.stream().on('log', (log: LogInfo) => {
output.unshift(log);
output = output.slice(0, 301);
if(isLogLineMinLevel(log, logConfig.level)) {
io.emit('log', formatLogToHtml(log[MESSAGE]));
}
});
if(appConfigFail) {
logger.warn('App config file exists but could not be parsed!');
}
const root = createRoot();
const localUrl = root.get('localUrl');
const notifiers = root.get('notifiers');
await notifiers.buildWebhooks(webhooks);
const availableLevels = ['error', 'warn', 'info', 'verbose', 'debug'];
/*
* setup clients
* */
const scrobbleClients = new ScrobbleClients(configDir);
await scrobbleClients.buildClientsFromConfig();
const scrobbleClients = root.get('clients');
await scrobbleClients.buildClientsFromConfig(notifiers);
if (scrobbleClients.clients.length === 0) {
logger.warn('No scrobble clients were configured!')
}
const scrobbleSources = new ScrobbleSources(localUrl, configDir);
await scrobbleSources.buildSourcesFromConfig();
const scrobbleSources = root.get('sources');//new ScrobbleSources(localUrl, configDir);
await scrobbleSources.buildSourcesFromConfig([]);
const clientCheckMiddle = makeClientCheckMiddle(scrobbleClients);
const sourceCheckMiddle = makeSourceCheckMiddle(scrobbleSources);
@@ -170,7 +151,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
}
app.getAsync('/', async function (req, res) {
let slicedLog = output.slice(0, logConfig.limit + 1);
let slicedLog = output.filter(x => isLogLineMinLevel(x, logConfig.level)).slice(0, logConfig.limit + 1).map(x => formatLogToHtml(x[MESSAGE]));
if (logConfig.sort === 'ascending') {
slicedLog.reverse();
}
@@ -243,7 +224,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
output: slicedLog,
limit: [10, 20, 50, 100].map(x => `<a class="capitalize ${logConfig.limit === x ? 'font-bold no-underline pointer-events-none' : ''}" data-limit="${x}" href="logs/settings/update?limit=${x}">${x}</a>`).join(' | '),
sort: ['ascending', 'descending'].map(x => `<a class="capitalize ${logConfig.sort === x ? 'font-bold no-underline pointer-events-none' : ''}" data-sort="${x}" href="logs/settings/update?sort=${x}">${x}</a>`).join(' | '),
level: availableLevels.map(x => `<a class="capitalize log-${x} ${logConfig.level === x ? `font-bold no-underline pointer-events-none` : ''}" data-log="${x}" href="logs/settings/update?level=${x}">${x}</a>`).join(' | ')
level: availableLevels.map(x => `<a class="capitalize log-level log-${x} ${logConfig.level === x ? `font-bold no-underline pointer-events-none` : ''}" data-log="${x}" href="logs/settings/update?level=${x}">${x}</a>`).join(' | ')
}
});
})
@@ -252,7 +233,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
app.postAsync('/tautulli', async function(this: any, req, res) {
tauIngress.trackIngress(req, false);
const payload = TautulliSource.formatPlayObj(req.body, true);
const payload = TautulliSource.formatPlayObj(req.body, {newFromSource: true});
// try to get config name from payload
if (req.body.scrobblerConfig !== undefined) {
const source = scrobbleSources.getByName(req.body.scrobblerConfig);
@@ -262,7 +243,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
return res.send('OK');
} else {
// @ts-expect-error TS(2339): Property 'handle' does not exist on type 'never'.
await source.handle(payload, scrobbleClients);
await source.handle(payload);
return res.send('OK');
}
} else {
@@ -274,14 +255,14 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
const tSources = scrobbleSources.getByType('tautulli');
for (const source of tSources) {
// @ts-expect-error TS(2339): Property 'handle' does not exist on type 'never'.
await source.handle(payload, scrobbleClients);
await source.handle(payload);
}
res.send('OK');
});
const plexMiddle = plexRequestMiddle();
const plexLog = createLabelledLogger('plexReq', 'Plex Request');
const plexLog = logger.child({labels: ['Plex Request']}, mergeArr);
const plexIngress = new PlexNotifier();
app.postAsync('/plex',
async function (req, res, next) {
@@ -295,7 +276,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
const { payload } = req as any;
if(payload !== undefined) {
const playObj = PlexSource.formatPlayObj(payload, true);
const playObj = PlexSource.formatPlayObj(payload, {newFromSource: true});
const pSources = scrobbleSources.getByType('plex') as PlexSource[];
if(pSources.length === 0) {
@@ -303,7 +284,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
}
for (const source of pSources) {
await source.handle(playObj, scrobbleClients);
await source.handle(playObj);
}
}
@@ -323,10 +304,29 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
jellyfinJsonParser, async function (req, res) {
jellyIngress.trackIngress(req, false);
const playObj = JellyfinSource.formatPlayObj(req.body, true);
const parts = remoteHostIdentifiers(req);
const connectionId = `${parts.host}-${parts.proxy ?? ''}`;
const playObj = JellyfinSource.formatPlayObj({...req.body, connectionId}, {newFromSource: true});
const pSources = scrobbleSources.getByType('jellyfin') as JellyfinSource[];
if(pSources.length === 0) {
logger.warn('Received Jellyfin connection but no Jellyfin sources are configured');
}
const logPayload = pSources.some(x => {
const {
data: {
options: {
logPayload = false
} = {}
} = {},
} = x.config;
return logPayload;
});
if(logPayload) {
logger.debug(`[Jellyfin] Logging payload due to at least one Jellyfin source having 'logPayload: true`, req.body);
}
for (const source of pSources) {
await source.handle(playObj, scrobbleClients);
await source.handle(playObj);
}
res.send('OK');
});
@@ -387,7 +387,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
return res.status(400).send(`Specified source cannot poll (${source.type})`);
}
source.poll(scrobbleClients);
source.poll();
res.send('OK');
});
@@ -401,7 +401,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
return res.status(400).send(`Specified source cannot retrieve recent plays (${source.type})`);
}
const result = await (source as AbstractSource).getRecentlyPlayed({formatted: true});
const result = (source as AbstractSource).getFlatRecentlyDiscoveredPlays();
const artistTruncFunc = truncateStringToLength(Math.min(40, longestString(result.map((x: any) => x.data.artists.join(' / ')).flat())));
const trackLength = longestString(result.map((x: any) => x.data.track))
const plays = result.map((x: PlayObject) => {
@@ -438,14 +438,14 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
logConfig.sort = val as string;
break;
case 'level':
logConfig.level = val as string;
for (const [key, logger] of winston.loggers.loggers) {
logger.level = val as string;
}
logConfig.level = val as LogLevel;
// for (const [key, logger] of winston.loggers.loggers) {
// logger.level = val as string;
// }
break;
}
}
let slicedLog = output.slice(0, logConfig.limit + 1);
let slicedLog = output.filter(x => isLogLineMinLevel(x, logConfig.level)).slice(0, logConfig.limit + 1).map(x => formatLogToHtml(x[MESSAGE]));
if (logConfig.sort === 'ascending') {
slicedLog.reverse();
}
@@ -468,7 +468,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
return res.send('Error with deezer credentials storage');
} else if(entity.config.data.accessToken !== undefined) {
// start polling
entity.poll(entity.clients)
entity.poll()
return res.redirect('/');
} else {
await sleep(1500);
@@ -509,7 +509,7 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
const tokenResult = await source.handleAuthCodeCallback(req.query);
let responseContent = 'OK';
if (tokenResult === true) {
source.poll(scrobbleClients);
source.poll();
} else {
responseContent = tokenResult;
}
@@ -517,6 +517,19 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
}
});
app.getAsync('/health', async function (req, res) {
const {
type,
name
} = req.query;
const [sourcesReady, sourceMessages] = await scrobbleSources.getStatusSummary(type as string|undefined, name as string|undefined);
const [clientsReady, clientMessages] = await scrobbleClients.getStatusSummary(type as string|undefined, name as string|undefined);
return res.status((clientsReady && sourcesReady) ? 200 : 500).json({messages: sourceMessages.concat(clientMessages)});
});
app.useAsync(async function (req, res) {
const remote = req.connection.remoteAddress;
const proxyRemote = req.headers["x-forwarded-for"];
@@ -534,18 +547,18 @@ const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`)
if ((source as SpotifySource).spotifyApi.getAccessToken() === undefined) {
anyNotReady = true;
} else {
(source as SpotifySource).poll(scrobbleClients);
(source as SpotifySource).poll();
}
}
break;
case 'lastfm':
if(source.initialized === true) {
source.poll(scrobbleClients);
source.poll();
}
break;
default:
if (source.poll !== undefined) {
source.poll(scrobbleClients);
source.poll();
}
}
}
+39
View File
@@ -0,0 +1,39 @@
import {createContainer} from "iti";
import path from "path";
import {projectDir} from "./common/index.js";
import ScrobbleClients from "./clients/ScrobbleClients.js";
import ScrobbleSources from "./sources/ScrobbleSources.js";
import {Notifiers} from "./notifier/Notifiers.js";
import {EventEmitter} from "events";
import {logPath} from "./common/logging.js";
import {Container} from "winston";
//import ScrobbleClients from "./clients/ScrobbleClients.js";
const configDir = process.env.CONFIG_DIR || path.resolve(projectDir, `./config`);
const port = process.env.PORT ?? 9078;
/*let logPath = path.resolve(projectDir, `./logs`);
if(typeof process.env.CONFIG_DIR === 'string') {
logPath = path.resolve(process.env.CONFIG_DIR, './logs');
}*/
//let root: Container;
export const createRoot = () => {
return createContainer().add({
configDir: configDir,
logDir: logPath,
localUrl: `http://localhost:${port}`,
clientEmitter: () => new EventEmitter(),
sourceEmitter: () => new EventEmitter(),
notifierEmitter: () => new EventEmitter(),
}).add((items) => ({
clients: () => new ScrobbleClients(items.clientEmitter, items.sourceEmitter, items.configDir),
sources: () => new ScrobbleSources(items.sourceEmitter, items.localUrl, items.configDir),
notifiers: () => new Notifiers(items.notifierEmitter, items.clientEmitter, items.sourceEmitter),
}));
}
export default createRoot;
+41
View File
@@ -0,0 +1,41 @@
import {GotifyConfig, NtfyConfig, WebhookPayload} from "../common/infrastructure/config/health/webhooks.js";
import {Logger} from "winston";
import {mergeArr} from "../utils.js";
export abstract class AbstractWebhookNotifier {
config: GotifyConfig | NtfyConfig
logger: Logger;
initialized: boolean = false;
requiresAuth: boolean = false;
authed: boolean = false;
protected constructor(type: string, defaultName: string, config: GotifyConfig | NtfyConfig, logger: Logger) {
this.config = config;
const label = `${type} - ${config.name ?? defaultName}`
this.logger = logger.child({labels: [label]}, mergeArr);
}
initialize = async () => {
this.initialized = true;
this.logger.verbose('Initialized');
}
testAuth = async () => {
return;
}
notify = async (payload: WebhookPayload) => {
if(!this.initialized) {
this.logger.debug('Will not use notifier because it is not initialized.');
return;
}
if(this.requiresAuth && !this.authed) {
this.logger.debug('Will not use notifier because it is not correctly authenticated.');
return;
}
return await this.doNotify(payload);
}
abstract doNotify: (payload: WebhookPayload) => Promise<any>;
}
+67
View File
@@ -0,0 +1,67 @@
import {AbstractWebhookNotifier} from "./AbstractWebhookNotifier.js";
import {GotifyConfig, PrioritiesConfig, WebhookPayload} from "../common/infrastructure/config/health/webhooks.js";
import {gotify} from 'gotify';
import request from 'superagent';
import {HTTPError} from "got";
import {Logger} from "winston";
export class GotifyWebhookNotifier extends AbstractWebhookNotifier {
declare config: GotifyConfig;
priorities: PrioritiesConfig;
constructor(defaultName: string, config: GotifyConfig, logger: Logger) {
super('Gotify', defaultName, config, logger);
this.requiresAuth = true;
const {
info = 5,
warn = 7,
error = 10
} = this.config.priorities || {};
this.priorities = {
info,
warn,
error
}
}
initialize = async () => {
// check url is correct
try {
const url = this.config.url;
const resp = await request.get(`${url}/version`);
this.logger.verbose(`Initialized. Found Server version ${resp.body.version}`);
this.initialized = true;
} catch (e) {
this.logger.error(`Failed to contact server | Error: ${e.message}`);
}
}
testAuth = async () => {
this.authed = true;
// TODO no easy way to test token is working without also pushing a message -- instead will de-auth if we get the right error message when trying to push for the first time
}
doNotify = async (payload: WebhookPayload) => {
try {
await gotify({
server: this.config.url,
app: this.config.token,
message: payload.message,
title: payload.title,
priority: this.priorities[payload.priority]
});
this.logger.debug(`Pushed notification.`);
} catch (e: any) {
if(e instanceof HTTPError && e.response.statusCode === 401) {
this.logger.error(`Unable to push notification. Error returned with 401 which means the TOKEN provided is probably incorrect. Disabling Notifier | Error => ${e.response.body}`);
this.authed = false;
} else {
this.logger.error(`Failed to push notification | Error => ${e.message}`);
}
}
}
}
+65
View File
@@ -0,0 +1,65 @@
import winston, {config, format, Logger} from "winston";
import {mergeArr} from "../utils.js";
import {
GotifyConfig,
NtfyConfig,
WebhookConfig,
WebhookPayload
} from "../common/infrastructure/config/health/webhooks.js";
import {AbstractWebhookNotifier} from "./AbstractWebhookNotifier.js";
import {GotifyWebhookNotifier} from "./GotifyWebhookNotifier.js";
import {NtfyWebhookNotifier} from "./NtfyWebhookNotifier.js";
import {EventEmitter} from "events";
export class Notifiers {
logger: Logger;
webhooks: AbstractWebhookNotifier[] = [];
emitter: EventEmitter;
clientEmitter: EventEmitter;
sourceEmitter: EventEmitter;
constructor(emitter: EventEmitter, clientEmitter: EventEmitter, sourceEmitter: EventEmitter) {
this.emitter = emitter;
this.clientEmitter = clientEmitter;
this.sourceEmitter = sourceEmitter;
this.logger = winston.loggers.get('app').child({labels: ['Notifiers']}, mergeArr);
this.sourceEmitter.on('notify', async (payload: WebhookPayload) => {
await this.notify(payload);
})
}
buildWebhooks = async (webhookConfigs: WebhookConfig[]) => {
for (const [i, config] of Object.entries(webhookConfigs)) {
let webhook: AbstractWebhookNotifier;
const defaultName = `Config ${i}`
switch (config.type) {
case 'gotify':
webhook = new GotifyWebhookNotifier(defaultName, config as GotifyConfig, this.logger);
break;
case 'ntfy':
webhook = new NtfyWebhookNotifier(defaultName, config as NtfyConfig, this.logger);
break;
default:
this.logger.error(`'${config.type}' is not a valid webhook type`);
continue;
}
await webhook.initialize();
if(webhook.initialized) {
await webhook.testAuth();
}
this.webhooks.push(webhook);
}
}
notify = async (payload: WebhookPayload) => {
for (const webhook of this.webhooks) {
await webhook.notify(payload);
}
}
}
+78
View File
@@ -0,0 +1,78 @@
import {AbstractWebhookNotifier} from "./AbstractWebhookNotifier.js";
import {
NtfyConfig,
PrioritiesConfig,
WebhookPayload
} from "../common/infrastructure/config/health/webhooks.js";
import {publish} from 'ntfy';
import request from "superagent";
import {Logger} from "winston";
export class NtfyWebhookNotifier extends AbstractWebhookNotifier {
declare config: NtfyConfig;
priorities: PrioritiesConfig;
constructor(defaultName: string, config: NtfyConfig, logger: Logger) {
super('Ntfy', defaultName, config, logger);
const {
info = 3,
warn = 4,
error = 5
} = this.config.priorities || {};
this.priorities = {
info,
warn,
error
}
}
initialize = async () => {
// check url is correct
try {
const url = this.config.url;
const resp = await request.get(`${url}/v1/health`);
if(resp.body !== undefined && typeof resp.body === 'object') {
const {health} = resp.body;
if(health === false) {
this.logger.error('Found Ntfy server but it responded that it was not ready.')
return;
}
} else {
this.logger.error(`Found Ntfy server but expected a response with 'health' in payload. Found => ${resp.body}`);
return;
}
this.logger.info('Initialized. Found Ntfy server');
this.initialized = true;
} catch (e) {
this.logger.error(`Failed to contact Ntfy server | Error: ${e.message}`);
}
}
doNotify = async (payload: WebhookPayload) => {
try {
let authorization = {};
if (this.config.username !== undefined) {
authorization = {
username: this.config.username,
password: this.config.password,
}
}
await publish({
message: payload.message,
topic: this.config.topic,
title: payload.title,
server: this.config.url,
priority: this.priorities[payload.priority],
...authorization,
});
this.logger.debug(`Pushed notification.`);
} catch (e: any) {
this.logger.error(`Failed to push notification: ${e.message}`)
}
}
}
+162 -99
View File
@@ -1,12 +1,29 @@
import dayjs, {Dayjs} from "dayjs";
import {buildTrackString, capitalize, createLabelledLogger, sleep} from "../utils.js";
import {InternalConfig, PlayObject, SourceType} from "../common/infrastructure/Atomic.js";
import {
buildTrackString,
capitalize, closePlayDate,
genGroupId, mergeArr,
playObjDataMatch, pollingBackoff,
sleep, sortByNewestPlayDate,
sortByOldestPlayDate
} from "../utils.js";
import {
GroupedFixedPlays,
GroupedPlays,
InternalConfig,
PlayObject, ProgressAwarePlayObject,
SourceType
} from "../common/infrastructure/Atomic.js";
import {Logger} from "winston";
import {SourceConfig} from "../common/infrastructure/config/source/sources.js";
import {EventEmitter} from "events";
import {FixedSizeList} from "fixed-size-list";
export interface RecentlyPlayedOptions {
limit?: number
formatted?: boolean
display?: boolean
}
export default abstract class AbstractSource {
@@ -19,11 +36,14 @@ export default abstract class AbstractSource {
clients: string[];
logger: Logger;
instantiatedAt: Dayjs;
lastActivityAt: Dayjs;
initialized: boolean = false;
requiresAuth: boolean = false;
requiresAuthInteraction: boolean = false;
authed: boolean = false;
multiPlatform: boolean = false;
localUrl: string;
configDir: string;
@@ -33,17 +53,23 @@ export default abstract class AbstractSource {
pollRetries: number = 0;
tracksDiscovered: number = 0;
constructor(type: SourceType, name: string, config: SourceConfig, internal: InternalConfig) {
emitter: EventEmitter;
protected recentDiscoveredPlays: GroupedFixedPlays = new Map();
constructor(type: SourceType, name: string, config: SourceConfig, internal: InternalConfig, emitter: EventEmitter) {
const {clients = [] } = config;
this.type = type;
this.name = name;
this.identifier = `Source - ${capitalize(this.type)} - ${name}`;
this.logger = createLabelledLogger(this.identifier, this.identifier);
this.logger = internal.logger.child({labels: [`${capitalize(this.type)} - ${name}`]}, mergeArr);
this.config = config;
this.clients = clients;
this.instantiatedAt = dayjs();
this.lastActivityAt = this.instantiatedAt;
this.localUrl = internal.localUrl;
this.configDir = internal.configDir;
this.emitter = emitter;
}
// default init function, should be overridden if init stage is required
@@ -64,17 +90,88 @@ export default abstract class AbstractSource {
// by default if the track was recently played it is valid
// this is useful for sources where the track doesn't have complete information like Subsonic
// TODO make this more descriptive? or move it elsewhere
recentlyPlayedTrackIsValid = (playObj: any) => {
recentlyPlayedTrackIsValid = (playObj: PlayObject) => {
return true;
}
poll = async (allClients: any) => {
await this.startPolling(allClients);
protected addPlayToDiscovered = (play: PlayObject) => {
const platformId = this.multiPlatform ? genGroupId(play) : 'SingleUser';
const list = this.recentDiscoveredPlays.get(platformId) ?? new FixedSizeList<ProgressAwarePlayObject>(30);
list.add(play);
this.recentDiscoveredPlays.set(platformId, list);
this.tracksDiscovered++;
this.logger.info(`Discovered => ${buildTrackString(play)}`);
}
startPolling = async (allClients: any) => {
if(this.requiresAuthInteraction && !this.authed) {
this.logger.error('Cannot start polling because user interaction is required for authentication');
getFlatRecentlyDiscoveredPlays = (): PlayObject[] => {
return Array.from(this.recentDiscoveredPlays.values()).map(x => x.data).flat(3).sort(sortByNewestPlayDate);
}
getRecentlyDiscoveredPlaysByPlatform = (platformId): PlayObject[] => {
const list = this.recentDiscoveredPlays.get(platformId);
if (list !== undefined) {
const data = [...list.data];
data.sort(sortByOldestPlayDate);
return data;
}
return [];
}
existingDiscovered = (play: PlayObject): PlayObject | undefined => {
const list = this.getRecentlyDiscoveredPlaysByPlatform(this.multiPlatform ? genGroupId(play) : 'SingleUser');
return list.find(x => playObjDataMatch(x, play) && closePlayDate(x, play));
}
alreadyDiscovered = (play: PlayObject): boolean => {
return this.existingDiscovered(play) !== undefined;
}
protected scrobble = (plays: PlayObject[], options: { forceRefresh?: boolean } = {}) => {
const newDiscoveredPlays: PlayObject[] = [];
for(const play of plays) {
if(!this.alreadyDiscovered(play)) {
this.addPlayToDiscovered(play);
newDiscoveredPlays.push(play);
}
}
if(newDiscoveredPlays.length > 0) {
newDiscoveredPlays.sort(sortByOldestPlayDate);
this.emitter.emit('scrobble', {
data: newDiscoveredPlays,
options: {
...options,
checkTime: newDiscoveredPlays[newDiscoveredPlays.length-1].data.playDate.add(2, 'second'),
scrobbleFrom: this.identifier,
scrobbleTo: this.clients
}
});
}
return newDiscoveredPlays;
}
protected notify = (payload) => {
this.emitter.emit('notify', payload);
}
poll = async () => {
await this.startPolling();
}
startPolling = async () => {
if(this.requiresAuth && !this.authed) {
if(this.requiresAuthInteraction) {
this.notify({title: `${this.identifier} - Polling Error`, message: 'Cannot start polling because user interaction is required for authentication', priority: 'error'});
this.logger.error('Cannot start polling because user interaction is required for authentication');
} else {
this.notify( {title: `${this.identifier} - Polling Error`, message: 'Cannot start polling because source does not have authentication.', priority: 'error'});
this.logger.error('Cannot start polling because source is not authenticated correctly.');
}
return;
}
// reset poll attempts if already previously run
@@ -82,137 +179,103 @@ export default abstract class AbstractSource {
const {
data: {
maxPollRetries = 0,
retryMultiplier = 1.5,
maxPollRetries = 5,
retryMultiplier = 1,
} = {},
} = this.config.data;
} = this.config;
// can't have negative retries!
const maxRetries = Math.max(0, maxPollRetries);
while (this.pollRetries <= maxRetries) {
try {
await this.doPolling(allClients);
await this.doPolling();
} catch (e) {
if (this.pollRetries < maxRetries) {
const delayFor = (this.pollRetries + 1) * retryMultiplier;
this.logger.info(`Poll reties (${this.pollRetries}) less than max poll retries (${maxRetries}), restarting polling after ${delayFor} second delay...`);
const delayFor = pollingBackoff(this.pollRetries + 1, retryMultiplier);
this.logger.info(`Poll retries (${this.pollRetries}) less than max poll retries (${maxRetries}), restarting polling after ${delayFor} second delay...`);
this.notify({title: `${this.identifier} - Polling Retry`, message: `Encountered error while polling but retries (${this.pollRetries}) are less than max poll retries (${maxRetries}), restarting polling after ${delayFor} second delay. | Error: ${e.message}`, priority: 'warn'});
await sleep((delayFor) * 1000);
} else {
this.logger.warn(`Poll retries (${this.pollRetries}) equal to max poll retries (${maxRetries}), stopping polling!`);
this.notify({title: `${this.identifier} - Polling Error`, message: `Encountered error while polling and retries (${this.pollRetries}) are equal to max poll retries (${maxRetries}), stopping polling!. | Error: ${e.message}`, priority: 'error'});
}
this.pollRetries++;
}
}
}
/**
* @param {ScrobbleClients} allClients
*/
doPolling = async (allClients: any) => {
doPolling = async () => {
if (this.polling === true) {
return;
}
this.logger.info('Polling started');
let lastTrackPlayedAt = this.instantiatedAt;
this.notify({title: `${this.identifier} - Polling Started`, message: 'Polling Started', priority: 'info'});
this.lastActivityAt = dayjs();
let checkCount = 0;
let checksOverThreshold = 0;
const {interval = 30, checkActiveFor = 300, maxInterval = 60} = this.config.data;
const maxBackoff = maxInterval - interval;
let sleepTime = interval;
try {
this.polling = true;
while (true) {
// @ts-expect-error TS(2367): This condition will always return 'false' since th... Remove this comment to see the full error message
// @ts-ignore
if(this.polling === false) {
this.logger.info('Stopped polling due to user input');
break;
}
let playObjs: PlayObject[] = [];
this.logger.debug('Refreshing recently played')
playObjs = await this.getRecentlyPlayed({formatted: true});
checkCount++;
let newTracksFound = false;
let closeToInterval = false;
const now = dayjs();
this.logger.debug('Refreshing recently played');
const playObjs = await this.getRecentlyPlayed({formatted: true});
const playInfo = playObjs.reduce((acc, playObj) => {
if(this.recentlyPlayedTrackIsValid(playObj)) {
const {data: {
playDate
} = {}
} = playObj;
if (playDate.unix() > lastTrackPlayedAt.unix()) {
newTracksFound = true;
this.logger.info(`New Track => ${buildTrackString(playObj)}`);
if (closeToInterval === false) {
closeToInterval = Math.abs(now.unix() - playDate.unix()) < 5;
}
return {
plays: [...acc.plays, {...playObj, meta: {...playObj.meta, newFromSource: true}}],
lastTrackPlayedAt: playDate
}
}
return {
...acc,
plays: [...acc.plays, playObj]
}
}
return acc;
}, {plays: [], lastTrackPlayedAt});
playObjs = playInfo.plays;
lastTrackPlayedAt = playInfo.lastTrackPlayedAt;
if (closeToInterval) {
// because the interval check was so close to the play date we are going to delay client calls for a few secs
// this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
// additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
this.logger.info('Track is close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first');
await sleep(10 * 1000);
}
if (newTracksFound === false) {
if (playObjs.length === 0) {
this.logger.debug(`No new tracks found and no tracks returned from API`);
} else {
this.logger.debug(`No new tracks found. Newest track returned was ${buildTrackString(playObjs.slice(-1)[0])}`);
}
} else {
checkCount = 0;
checksOverThreshold = 0;
}
let scrobbleResult = [];
let newDiscovered: PlayObject[] = [];
if(playObjs.length > 0) {
// use the source instantiation time or the last track play time to determine if we should refresh clients...
// we only need to refresh clients when the source has "newer" information otherwise we're just refreshing clients for no reason
scrobbleResult = await allClients.scrobble(playObjs, {
checkTime: lastTrackPlayedAt.add(2, 's'),
forceRefresh: closeToInterval,
scrobbleFrom: this.identifier,
scrobbleTo: this.clients
});
const now = dayjs().unix();
const closeToInterval = playObjs.some(x => now - x.data.playDate.unix() < 5);
if (playObjs.length > 0 && closeToInterval) {
// because the interval check was so close to the play date we are going to delay client calls for a few secs
// this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
// additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
this.logger.info('Potential plays were discovered close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first');
await sleep(10 * 1000);
}
newDiscovered = this.scrobble(playObjs,
{
forceRefresh: closeToInterval
});
}
if (scrobbleResult.length > 0) {
if(newDiscovered.length > 0) {
// only update date if the play date is after the current activity date (in the case of backlogged plays)
this.lastActivityAt = newDiscovered[0].data.playDate.isAfter(this.lastActivityAt) ? newDiscovered[0].data.playDate : this.lastActivityAt;
checkCount = 0;
this.tracksDiscovered += scrobbleResult.length;
checksOverThreshold = 0;
} else {
this.logger.debug(`No new tracks discovered`);
}
const {interval = 30, checkActiveFor = 300, maxSleep = 300} = this.config.data;
let sleepTime = interval;
// don't need to do back off calc if interval is 5 minutes or greater since its already pretty light on API calls
// and don't want to back off if we just started the app
const activeThreshold = lastTrackPlayedAt.add(checkActiveFor, 's');
if (activeThreshold.isBefore(dayjs()) && sleepTime < 300) {
const activeThreshold = this.lastActivityAt.add(checkActiveFor, 's');
const inactiveFor = dayjs.duration(Math.abs(activeThreshold.diff(dayjs(), 'millisecond'))).humanize(false);
if (activeThreshold.isBefore(dayjs())) {
checksOverThreshold++;
const backoffMultiplier = Math.min(checksOverThreshold, 1000) * 1.5;
sleepTime = Math.min(interval * backoffMultiplier, maxSleep);
if(sleepTime < maxInterval) {
const checkVal = Math.min(checksOverThreshold, 1000);
const backoff = Math.round(Math.max(Math.min(Math.min(checkVal, 1000) * 2 * (1.1 * checkVal), maxBackoff), 5));
sleepTime = interval + backoff;
this.logger.debug(`Last activity was at ${this.lastActivityAt.format()} which is ${inactiveFor} outside of active polling period of (last activity + ${checkActiveFor} seconds). Will sleep for interval ${interval} + ${backoff} seconds.`);
} else {
this.logger.debug(`Last activity was at ${this.lastActivityAt.format()} which is ${inactiveFor} outside of active polling period of (last activity + ${checkActiveFor} seconds). Will sleep for max interval ${maxInterval} seconds.`);
}
} else {
sleepTime = interval;
this.logger.debug(`Last activity was at ${this.lastActivityAt.format()}. Will sleep for interval ${sleepTime} seconds.`);
}
// sleep for interval
this.logger.debug(`Sleeping for ${sleepTime}s`);
this.logger.verbose(`Sleeping for ${sleepTime}s`);
await sleep(sleepTime * 1000);
}
+9 -12
View File
@@ -1,10 +1,11 @@
import request from 'superagent';
import {parseRetryAfterSecsFromObj, readJson, sleep, sortByPlayDate, writeFile} from "../utils.js";
import {parseRetryAfterSecsFromObj, readJson, sleep, sortByOldestPlayDate, writeFile} from "../utils.js";
import {Strategy as DeezerStrategy} from 'passport-deezer';
import AbstractSource, {RecentlyPlayedOptions} from "./AbstractSource.js";
import dayjs from "dayjs";
import {DeezerSourceConfig} from "../common/infrastructure/config/source/deezer.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import EventEmitter from "events";
export default class DeezerSource extends AbstractSource {
workingCredsPath;
@@ -17,8 +18,8 @@ export default class DeezerSource extends AbstractSource {
declare config: DeezerSourceConfig;
constructor(name: any, config: DeezerSourceConfig, internal: InternalConfig) {
super('deezer', name, config, internal);
constructor(name: any, config: DeezerSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('deezer', name, config, internal, emitter);
const {
data: {
interval = 60,
@@ -35,7 +36,8 @@ export default class DeezerSource extends AbstractSource {
this.canPoll = true;
}
static formatPlayObj(obj: any, newFromSource = false): PlayObject {
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
title: name,
artist: {
@@ -60,9 +62,8 @@ export default class DeezerSource extends AbstractSource {
playDate: dayjs(timestamp * 1000),
},
meta: {
trackLength: duration,
source: 'Deezer',
sourceId: id,
trackId: id,
newFromSource,
url: {
web: link
@@ -101,12 +102,8 @@ export default class DeezerSource extends AbstractSource {
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
const {formatted = false} = options;
const resp = await this.callApi(request.get(`${this.baseUrl}/user/me/history`));
if(formatted) {
return resp.data.map((x: any) => DeezerSource.formatPlayObj(x)).sort(sortByPlayDate);
}
return resp.data;
return resp.data.map((x: any) => DeezerSource.formatPlayObj(x)).sort(sortByOldestPlayDate);
}
callApi = async (req: any, retries = 0) => {
+146
View File
@@ -0,0 +1,146 @@
import MemorySource from "./MemorySource.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import dayjs from "dayjs";
import {URL} from "url";
import normalizeUrl from 'normalize-url';
import {EventEmitter} from "events";
import {RecentlyPlayedOptions} from "./AbstractSource.js";
import {JRiverSourceConfig} from "../common/infrastructure/config/source/jriver.js";
import {Info, JRiverApiClient, PLAYER_STATE} from "../apis/JRiverApiClient.js";
export class JRiverSource extends MemorySource {
declare config: JRiverSourceConfig;
url: URL;
client: JRiverApiClient;
clientReady: boolean = false;
constructor(name: any, config: JRiverSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
const {
data,
} = config;
const {
interval = 10,
maxInterval = 30,
...rest
} = data || {};
super('jriver', name, {...config, data: {interval, maxInterval, ...rest}}, internal, emitter);
const {
data: {
url = 'http://localhost:52199/MCWS/v1/'
} = {},
} = config;
this.url = JRiverSource.parseConnectionUrl(url);
this.client = new JRiverApiClient(name, {...data, url: this.url.toString()});
this.requiresAuth = true;
this.canPoll = true;
this.multiPlatform = true;
}
static parseConnectionUrl(val: string) {
const normal = normalizeUrl(val, {removeTrailingSlash: true, normalizeProtocol: true})
const url = new URL(normal);
if (url.port === null || url.port === '') {
url.port = '52199';
}
if (url.pathname === '/') {
url.pathname = '/MCWS/v1/';
} else if (url.pathname === '/MCWS/v1') {
url.pathname = '/MCWS/v1/';
}
return url;
}
initialize = async () => {
const {
data: {
url
} = {}
} = this.config;
this.logger.debug(`Config URL: '${url ?? '(None Given)'}' => Normalized: '${this.url.toString()}'`)
const connected = await this.client.testConnection();
if(connected) {
this.logger.info('Connection OK');
this.initialized = true;
return true;
} else {
this.logger.error(`Could not connect.`);
this.initialized = false;
return false;
}
}
testAuth = async () => {
const resp = await this.client.testAuth();
this.authed = resp;
this.clientReady = this.authed;
return this.authed;
}
static formatPlayObj(obj: Info, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = true} = options;
const {
Artist,
Album,
Name,
DurationMS,
PositionMS: trackProgressPosition,
FileKey,
ZoneID,
ZoneName,
} = obj;
let artists = Artist === null || Artist === undefined ? [] : [Artist];
let album = Album === null || Album === '' ? undefined : Album;
const length = Number.parseInt(DurationMS.toString()) / 1000;
return {
data: {
track: Name,
album: album,
artists,
duration: Math.round(length),
playDate: dayjs()
},
meta: {
source: 'mopidy',
trackId: FileKey,
newFromSource,
trackProgressPosition: trackProgressPosition !== undefined ? Math.round(Number.parseInt(trackProgressPosition.toString()) / 1000) : undefined,
deviceId: `Zone${ZoneID}${ZoneName !== undefined ? `-${ZoneName}` : ''}`,
}
}
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
if (!this.clientReady) {
this.logger.warn('Cannot actively poll since client is not connected.');
return [];
}
let play = [];
//should it use zones?
//const zoneResp = await this.client.getZones();
const infoResp = await this.client.getInfo();
const {
body: {
data,
} = {}
} = infoResp;
if(data !== undefined) {
const {State} = data;
if(State !== PLAYER_STATE.STOPPED) {
play = [JRiverSource.formatPlayObj(data)];
}
}
return this.processRecentPlays(play);
}
}
+59 -48
View File
@@ -1,18 +1,22 @@
import MemorySource from "./MemorySource.js";
import dayjs from "dayjs";
import {buildTrackString, parseDurationFromTimestamp} from "../utils.js";
import {buildTrackString, combinePartsToString, parseDurationFromTimestamp, truncateStringToLength} from "../utils.js";
import {JellySourceConfig} from "../common/infrastructure/config/source/jellyfin.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import EventEmitter from "events";
const shortDeviceId = truncateStringToLength(10, '');
export default class JellyfinSource extends MemorySource {
users;
servers;
multiPlatform: boolean = true;
declare config: JellySourceConfig;
constructor(name: any, config: JellySourceConfig, internal: InternalConfig) {
super('jellyfin', name, config, internal);
constructor(name: any, config: JellySourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('jellyfin', name, config, internal, emitter);
const {data: {users, servers} = {}} = config;
if (users === undefined || users === null) {
@@ -45,7 +49,8 @@ export default class JellyfinSource extends MemorySource {
this.initialized = true;
}
static formatPlayObj(obj: any, newFromSource = false): PlayObject {
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
ServerId,
ServerName,
@@ -61,13 +66,30 @@ export default class JellyfinSource extends MemorySource {
ItemId,
ItemType,
PlaybackPosition,
connectionId,
DeviceId = '',
DeviceName,
ClientName,
} = obj;
const dur = parseDurationFromTimestamp(RunTime);
let server = ServerName;
if(server === undefined || server === '') {
server = ServerId;
}
if(server === undefined || server === '') {
server = connectionId;
}
let artists = [];
if(Artist !== undefined) {
artists = [Artist];
}
return {
data: {
artists: [Artist],
artists,
album: Album,
track: Name,
duration: dur !== undefined ? dur.as('seconds') : undefined,
@@ -76,13 +98,14 @@ export default class JellyfinSource extends MemorySource {
meta: {
event: NotificationType,
mediaType: ItemType,
sourceId: ItemId,
user: NotificationUsername,
server: ServerName,
trackId: ItemId,
user: NotificationUsername ?? UserId,
server,
source: 'Jellyfin',
newFromSource,
playbackPosition: parseDurationFromTimestamp(PlaybackPosition),
sourceVersion: ServerVersion
trackProgressPosition: PlaybackPosition !== undefined ? parseDurationFromTimestamp(PlaybackPosition).asSeconds() : undefined,
sourceVersion: ServerVersion,
deviceId: combinePartsToString([shortDeviceId(DeviceId), DeviceName])
}
}
}
@@ -98,11 +121,31 @@ export default class JellyfinSource extends MemorySource {
} = {}
} = playObj;
if (event !== undefined && !['PlaybackProgress','PlaybackStarted'].includes(event)) {
this.logger.debug(`Will not scrobble event because event type is not PlaybackProgress or PlaybackStarted, found event: ${event}`)
return false;
}
if (mediaType !== 'Audio') {
this.logger.debug(`Will not scrobble event because media type was not 'Audio', found type: ${mediaType}`, {
track
});
return false;
}
if (this.servers !== undefined && !this.servers.includes(server.toLocaleLowerCase())) {
this.logger.warn(`Will not scrobble event because server was not on allowed list, found server: ${server}`, {
track
})
return false;
}
if (this.users !== undefined) {
if (user === undefined) {
this.logger.warn(`Config defined users but payload contained no user info`);
this.logger.warn(`Will not scrobble event because config defined users but payload contained no user info`);
return false;
} else if (!this.users.includes(user.toLocaleLowerCase())) {
this.logger.debug(`Will not scrobble event because author was not an allowed user: ${user}`, {
this.logger.warn(`Will not scrobble event because author was not an allowed user: ${user}`, {
artists,
track
})
@@ -110,55 +153,23 @@ export default class JellyfinSource extends MemorySource {
}
}
if (event !== undefined && !['PlaybackProgress','PlaybackStarted'].includes(event)) {
this.logger.debug(`Will not scrobble event because it is not media.scrobble (${event})`, {
artists,
track
})
return false;
}
if (mediaType !== 'Audio') {
this.logger.debug(`Will not scrobble event because media type was not 'Audio' (${mediaType})`, {
artists,
track
});
return false;
}
if (this.servers !== undefined && !this.servers.includes(server.toLocaleLowerCase())) {
this.logger.debug(`Will not scrobble event because server was not on allowed list: ${server}`, {
artists,
track
})
return false;
}
return true;
}
getRecentlyPlayed = async (options = {}) => {
return this.statefulRecentlyPlayed;
return this.getFlatRecentlyDiscoveredPlays();
}
handle = async (playObj: any, allClients: any) => {
handle = async (playObj: any) => {
if (!this.isValidEvent(playObj)) {
return;
}
const newPlays = this.processRecentPlays([playObj]);
for(const p of newPlays) {
this.logger.info(`New Track => ${buildTrackString(p)}`);
}
if(newPlays.length > 0) {
const recent = await this.getRecentlyPlayed();
const newestPlay = recent[recent.length - 1];
try {
await allClients.scrobble(newPlays, {scrobbleTo: this.clients, scrobbleFrom: this.identifier, checkTime: newestPlay.data.playDate});
// only gets hit if we scrobbled ok
this.tracksDiscovered++;
this.scrobble(newPlays);
} catch (e) {
this.logger.error('Encountered error while scrobbling')
this.logger.error(e)
+75
View File
@@ -0,0 +1,75 @@
import MemorySource from "./MemorySource.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {EventEmitter} from "events";
import {RecentlyPlayedOptions} from "./AbstractSource.js";
import {KodiSourceConfig} from "../common/infrastructure/config/source/kodi.js";
import {KodiApiClient} from "../apis/KodiApiClient.js";
export class KodiSource extends MemorySource {
declare config: KodiSourceConfig;
client: KodiApiClient;
clientReady: boolean = false;
constructor(name: any, config: KodiSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
const {
data,
} = config;
const {
interval = 10,
maxInterval = 30,
...rest
} = data || {};
super('kodi', name, {...config, data: {interval, maxInterval, ...rest}}, internal, emitter);
this.client = new KodiApiClient(name, data);
this.requiresAuth = true;
this.canPoll = true;
this.multiPlatform = true;
}
initialize = async () => {
const {
data: {
url
} = {}
} = this.config;
this.logger.debug(`Config URL: '${url ?? '(None Given)'}' => Normalized: '${this.client.url.toString()}'`)
this.initialized = true;
return true;
/*const connected = await this.client.testConnection();
if(connected) {
//this.logger.info('Connection OK');
this.initialized = true;
return true;
} else {
this.logger.error(`Could not connect.`);
this.initialized = false;
return false;
}*/
}
testAuth = async () => {
const resp = await this.client.testAuth();
this.authed = resp;
this.clientReady = this.authed;
return this.authed;
}
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
return KodiApiClient.formatPlayObj(obj, options);
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
if (!this.clientReady) {
this.logger.warn('Cannot actively poll since client is not connected.');
return [];
}
let play = await this.client.getRecentlyPlayed(options);
return this.processRecentPlays(play);
}
}
+9 -8
View File
@@ -1,9 +1,10 @@
import AbstractSource, {RecentlyPlayedOptions} from "./AbstractSource.js";
import LastfmApiClient from "../apis/LastfmApiClient.js";
import {sortByPlayDate} from "../utils.js";
import {sortByOldestPlayDate} from "../utils.js";
import {LastfmClientConfig} from "../common/infrastructure/config/client/lastfm.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {UserGetRecentTracksResponse} from "lastfm-node-client";
import EventEmitter from "events";
export default class LastfmSource extends AbstractSource {
@@ -13,14 +14,14 @@ export default class LastfmSource extends AbstractSource {
declare config: LastfmClientConfig;
constructor(name: any, config: LastfmClientConfig, internal: InternalConfig) {
super('lastfm', name, config, internal);
constructor(name: any, config: LastfmClientConfig, internal: InternalConfig, emitter: EventEmitter) {
super('lastfm', name, config, internal, emitter);
this.canPoll = true;
this.api = new LastfmApiClient(name, config.data);
this.api = new LastfmApiClient(name, {...config.data, configDir: internal.configDir});
}
static formatPlayObj(obj: any): PlayObject {
return LastfmApiClient.formatPlayObj(obj);
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
return LastfmApiClient.formatPlayObj(obj, options);
}
initialize = async () => {
@@ -79,6 +80,6 @@ export default class LastfmSource extends AbstractSource {
this.logger.debug(x);
return acc;
}
}, []).sort(sortByPlayDate);
}, []).sort(sortByOldestPlayDate);
}
}
+57
View File
@@ -0,0 +1,57 @@
import AbstractSource, {RecentlyPlayedOptions} from "./AbstractSource.js";
import {FormatPlayObjectOptions, INITIALIZING, InternalConfig} from "../common/infrastructure/Atomic.js";
import EventEmitter from "events";
import {ListenBrainzSourceConfig} from "../common/infrastructure/config/source/listenbrainz.js";
import {ListenbrainzApiClient} from "../apis/ListenbrainzApiClient.js";
export default class ListenbrainzSource extends AbstractSource {
api: ListenbrainzApiClient;
requiresAuth = true;
requiresAuthInteraction = false;
declare config: ListenBrainzSourceConfig;
constructor(name: any, config: ListenBrainzSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('listenbrainz', name, config, internal, emitter);
this.canPoll = true;
this.api = new ListenbrainzApiClient(name, config.data);
}
static formatPlayObj = (obj: any, options: FormatPlayObjectOptions = {}) => ListenbrainzApiClient.formatPlayObj(obj, options);
initialize = async () => {
// @ts-expect-error TS(2322): Type 'number' is not assignable to type 'boolean'.
this.initialized = INITIALIZING;
if(this.config.data.token === undefined) {
this.logger.error('Must provide a User Token');
this.initialized = false;
} else {
try {
await this.api.testConnection();
this.initialized = true;
} catch (e) {
this.logger.error(e);
this.initialized = false;
}
}
return this.initialized;
}
testAuth = async () => {
try {
this.authed = await this.api.testAuth();
} catch (e) {
this.logger.error('Could not successfully communicate with Listenbrainz API');
this.logger.error(e);
this.authed = false;
}
return this.authed;
}
getRecentlyPlayed = async(options: RecentlyPlayedOptions = {}) => {
const {limit = 20} = options;
return await this.api.getRecentlyPlayed(limit);
}
}
+198
View File
@@ -0,0 +1,198 @@
import dbus, {ClientInterface, Variant} from 'dbus-next';
import dayjs from "dayjs";
import {
MPRIS_IFACE,
MPRIS_PATH,
MPRISMetadata, MPRISSourceConfig, PLAYBACK_STATUS_STOPPED,
PlaybackStatus, PlayerInfo,
PROPERTIES_IFACE
} from "../common/infrastructure/config/source/mpris.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import MemorySource from "./MemorySource.js";
import {RecentlyPlayedOptions} from "./AbstractSource.js";
import {removeDuplicates} from "../utils.js";
import EventEmitter from "events";
export class MPRISSource extends MemorySource {
declare config: MPRISSourceConfig;
whitelist: string[] = [];
blacklist: string[] = [];
multiPlatform: boolean = true;
constructor(name: any, config: MPRISSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('mpris', name, config, internal, emitter);
this.canPoll = true;
const {data: {whitelist = [], blacklist = []} = {}} = config;
if(!Array.isArray(whitelist)) {
this.whitelist = whitelist.split(',')
} else {
this.whitelist = whitelist;
}
if(!Array.isArray(blacklist)) {
this.blacklist = blacklist.split(',');
} else {
this.blacklist = blacklist;
}
}
static formatPlayObj(obj: PlayerInfo, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
name,
position,
metadata: {
length,
album,
artist = [],
albumArtist = [],
title,
trackid,
url,
} = {}
} = obj;
return {
data: {
track: title,
album,
artists: Array.from(new Set(artist.concat(albumArtist))),
duration: length,
playDate: dayjs()
},
meta: {
source: 'dbus',
trackId: trackid,
newFromSource,
url: {
web: url
},
trackProgressPosition: position,
deviceId: name,
}
}
}
initialize = async () => {
// test if we can get DBus
try {
await this.getDBus();
return true;
} catch (e) {
this.logger.error('Could not get DBus interface from operating system');
this.logger.error(e);
return false;
}
}
protected getDBus = async () => {
const bus = dbus.sessionBus();
const obj = await bus.getProxyObject('org.freedesktop.DBus', '/org/freedesktop/DBus');
return obj.getInterface('org.freedesktop.DBus');
}
protected listAll = async () => {
let iface = await this.getDBus();
let names = await iface.ListNames();
return names.filter((n) => n.startsWith('org.mpris.MediaPlayer2'))
}
getPlayersInfo = async (activeOnly = true): Promise<PlayerInfo[]> => {
const list = await this.listAll();
let bus = dbus.sessionBus();
const playerInfos: PlayerInfo[] = [];
for (const playerName of list) {
let obj = await bus.getProxyObject(playerName, MPRIS_PATH);
const plainPlayerName = playerName.replace('org.mpris.MediaPlayer2.', '');
//let player = obj.getInterface(MPRIS_IFACE);
let props = obj.getInterface(PROPERTIES_IFACE);
const pos = await this.getPlayerPosition(props);
const status = await this.getPlayerStatus(props);
if (status === PLAYBACK_STATUS_STOPPED && activeOnly) {
continue;
}
const metadata = await this.getPlayerMetadata(props);
playerInfos.push({
name: plainPlayerName,
status,
position: pos,
metadata
});
}
return playerInfos;
}
protected getPlayerPosition = async (props: ClientInterface): Promise<number> => {
const pos = await props.Get(MPRIS_IFACE, 'Position');
return dayjs.duration({milliseconds: Number(pos.value / 1000n)}).asSeconds();
}
protected getPlayerStatus = async (props: ClientInterface): Promise<PlaybackStatus> => {
const status = await props.Get(MPRIS_IFACE, 'PlaybackStatus');
return status.value as PlaybackStatus;
}
protected getPlayerMetadata = async (props: ClientInterface): Promise<MPRISMetadata> => {
const metadata = await props.Get(MPRIS_IFACE, 'Metadata');
return this.metadataToPlain(metadata.value);
}
metadataToPlain = (metadataVariant): MPRISMetadata => {
let metadataPlain = {};
for (let k of Object.keys(metadataVariant)) {
let value = metadataVariant[k];
if (value === undefined || value === null) {
//logging.warn(`ignoring a null metadata value for key ${k}`);
continue;
}
const plainKey = k.replace(/mpris:|xesam:/, '');
if (value instanceof Variant) {
if (typeof value.value === 'bigint') {
// in this context we're using it as a duration (track length or playback position)
metadataPlain[plainKey] = dayjs.duration({milliseconds: Number(value.value / 1000n)}).asSeconds();
} else {
metadataPlain[plainKey] = value.value;
}
} else {
metadataPlain[plainKey] = value;
}
}
return metadataPlain;
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
const infos = await this.getPlayersInfo();
let plays: PlayObject[] = [];
for(const info of infos) {
const lowerName = info.name.toLocaleLowerCase();
if(this.whitelist.length > 0) {
if(!this.whitelist.some(x => lowerName.includes(x.toLocaleLowerCase()))) {
this.logger.debug(`No name in whitelist was found in Player Name '${info.name}', skipping player`);
continue;
}
} else if(this.blacklist.length > 0) {
if(this.whitelist.some(x => lowerName.includes(x.toLocaleLowerCase()))) {
this.logger.debug(`A name in blacklist was found in Player Name '${info.name}', skipping player`);
continue;
}
}
plays.push(MPRISSource.formatPlayObj(info));
}
const deduped = removeDuplicates(plays);
if(options.display === true) {
return deduped;
}
return this.processRecentPlays(deduped);
}
}
+138 -58
View File
@@ -1,6 +1,14 @@
import AbstractSource from "./AbstractSource.js";
import {playObjDataMatch, sortByPlayDate, buildTrackString} from "../utils.js";
import {
playObjDataMatch,
sortByOldestPlayDate,
buildTrackString,
toProgressAwarePlayObject,
getProgress,
genGroupId
} from "../utils.js";
import dayjs from "dayjs";
import {GroupedPlays, PlayObject, ProgressAwarePlayObject} from "../common/infrastructure/Atomic.js";
export default class MemorySource extends AbstractSource {
/*
@@ -11,77 +19,149 @@ export default class MemorySource extends AbstractSource {
* * where these timestamps don't have enough granularity (IE second accuracy)
* such as subsonic and jellyfin */
statefulRecentlyPlayed = [];
candidateRecentlyPlayed = [];
/**
* Tracks we are tracked that we are confident qualified as being played based on:
*
* - MS saw the track for the first time while it was running/polling
* - Continued to see this same track through consecutive polling/ingress events for AT LEAST 30 seconds
* - If the play info contains playback position data we also check if that has progressed at least 30 seconds
*
* These are tracks that are actually used by the source to scrobble to clients
* */
//statefulRecentlyPlayed: GroupedPlays = new Map();
/**
* Tracks that are actively being tracked (discovered from source recently) to see if they will "qualify" as being played.
*
* Once a track qualifies it is added to statefuls.
* */
candidateRecentlyPlayed: GroupedPlays = new Map();
processRecentPlays = (plays: any) => {
// getFlatStatefulRecentlyPlayed = (): PlayObject[] => {
// return Array.from(this.statefulRecentlyPlayed.values()).flat().sort(sortByPlayDate);
// }
let newStatefulPlays: any = [];
// first format new plays with locked play date
const lockedPlays = plays.map((p: any) => {
processRecentPlays = (plays: PlayObject[], useExistingPlayDate = false) => {
let newStatefulPlays: PlayObject[] = [];
// if we can't trust existing play dates (like for subsonic where there is no timestamp) then
// we format new plays with locked play date IE the first time we have "seen" this track
// -- otherwise, for sources like Spotify that accurately report when track started to play, we can use existing dates
const flatLockedPlays = useExistingPlayDate ? plays : plays.map((p: any) => {
const {data: {playDate, ...restData}, ...rest} = p;
return {data: {...restData, playDate: dayjs()}, ...rest};
})
// if no candidates exist new plays are new candidates
if(this.candidateRecentlyPlayed.length === 0) {
for(const p of lockedPlays) {
this.logger.debug(`No prior candidate recent plays! Adding new locked plays: ${buildTrackString(p, {include: ['sourceId', 'artist', 'track']})}`);
}
this.candidateRecentlyPlayed = lockedPlays;
} else {
// otherwise determine new tracks (not found in prior candidates)
const newTracks = lockedPlays.filter((x: any) => this.candidateRecentlyPlayed.every(y => !playObjDataMatch(y, x)));
if(newTracks.length > 0) {
for(const p of newTracks) {
this.logger.debug(`New play found that does not match existing candidates will be added: ${buildTrackString(p, {include: ['sourceId', 'artist', 'track']})}`);
}
}
// filter prior candidates based on new recently played
this.candidateRecentlyPlayed = this.candidateRecentlyPlayed.filter(x => {
const candidateMatchedLocked = lockedPlays.some((y: any) => playObjDataMatch(x, y));
if(!candidateMatchedLocked) {
this.logger.debug(`Existing candidate not found in locked plays will be removed: ${buildTrackString(x, {include: ['sourceId', 'artist', 'track']})}`);
}
return candidateMatchedLocked;
});
// and then combine still playing with new tracks
this.candidateRecentlyPlayed = this.candidateRecentlyPlayed.concat(newTracks);
this.candidateRecentlyPlayed.sort(sortByPlayDate);
});
// group by device-user
const groupedLockedPlays = flatLockedPlays.reduce((acc: GroupedPlays, curr: ProgressAwarePlayObject) => {
const id = genGroupId(curr);
acc.set(id, (acc.get(id) ?? []).concat(curr));
return acc;
}, new Map());
for(const candidate of this.candidateRecentlyPlayed) {
const {data: {playDate, track}} = candidate;
if(playDate.isBefore(dayjs().subtract(30, 's'))) {
// a prior candidate has been playing for more than 30 seconds, time to check statefuls
for(const [groupId, lockedPlays] of groupedLockedPlays.entries()) {
let cRecentlyPlayed = this.candidateRecentlyPlayed.get(groupId) ?? [];
// if no candidates exist new plays are new candidates
if(cRecentlyPlayed.length === 0) {
this.logger.debug(`[Platform ${groupId}] No prior candidate recent plays!`)
// update activity date here so that polling interval decreases *before* we get a new valid play
// so that we don't miss a play due to long polling interval
this.lastActivityAt = dayjs();
const progressAware: ProgressAwarePlayObject[] = [];
for(const p of lockedPlays) {
progressAware.push(toProgressAwarePlayObject(p));
this.logger.debug(`[Platform ${groupId}] Adding new locked play: ${buildTrackString(p, {include: ['trackId', 'artist', 'track']})}`);
}
this.candidateRecentlyPlayed.set(groupId, progressAware)
} else {
// otherwise determine new tracks (not found in prior candidates)
const newTracks = lockedPlays.filter((x: any) => cRecentlyPlayed.every(y => !playObjDataMatch(y, x)));
const newProgressAwareTracks: ProgressAwarePlayObject[] = [];
if(newTracks.length > 0) {
// update activity date here so that polling interval decreases *before* we get a new valid play
// so that we don't miss a play due to long polling interval
this.lastActivityAt = dayjs();
this.logger.debug(`[Platform ${groupId}] New plays found that do not match existing candidates.`)
for(const p of newTracks) {
this.logger.debug(`[Platform ${groupId}] Adding new locked play: ${buildTrackString(p, {include: ['trackId', 'artist', 'track']})}`);
newProgressAwareTracks.push(toProgressAwarePlayObject(p));
}
}
// filter prior candidates based on new recently played
cRecentlyPlayed = cRecentlyPlayed.filter(x => {
const candidateMatchedLocked = lockedPlays.some((y: any) => playObjDataMatch(x, y));
if(!candidateMatchedLocked) {
this.logger.debug(`[Platform ${groupId}] Existing candidate not found in locked plays will be removed: ${buildTrackString(x, {include: ['trackId', 'artist', 'track']})}`);
}
return candidateMatchedLocked;
});
// and then combine still playing with new tracks
cRecentlyPlayed = cRecentlyPlayed.concat(newProgressAwareTracks);
cRecentlyPlayed.sort(sortByOldestPlayDate);
const matchingRecent = this.statefulRecentlyPlayed.find(x => playObjDataMatch(x, candidate));
let stPrefix = `(Stateful Play) ${buildTrackString(candidate, {include: ['sourceId', 'artist', 'track']})}`;
if(matchingRecent === undefined) {
this.logger.debug(`${stPrefix} added after being seen for 30 seconds and not matching any prior plays`);
newStatefulPlays.push(candidate);
this.statefulRecentlyPlayed.push(candidate);
} else {
const {data: { playDate, duration }} = candidate;
const {data: { playDate: rplayDate }} = matchingRecent;
if(!playDate.isSame(rplayDate)) {
if(duration !== undefined) {
if(playDate.isAfter(rplayDate.add(duration, 's'))) {
this.logger.debug(`${stPrefix} added after being seen for 30 seconds and having a different timestamp than a prior play`);
newStatefulPlays.push(candidate);
this.statefulRecentlyPlayed.push(candidate);
this.candidateRecentlyPlayed.set(groupId, cRecentlyPlayed);
//const sRecentlyPlayed = this.statefulRecentlyPlayed.get(groupId) ?? [];
// now we check if all candidates pass tests for having been tracked long enough:
// * Has been tracked for at least 30 seconds
// * If it has playback position data then it must also have progressed at least 30 seconds since our initial tracking data
for(const candidate of cRecentlyPlayed) {
const {data: {playDate, track}} = candidate;
const firstSeenValid = playDate.isBefore(dayjs().subtract(30, 's'));
let progressValid = firstSeenValid;
if (firstSeenValid) {
// check if we can get progress as well
const matchingLockedPlay = lockedPlays.find(x => playObjDataMatch(x, candidate));
// this should always be found but checking just in case
if (matchingLockedPlay !== undefined) {
const progress = getProgress(candidate, matchingLockedPlay);
if (progress !== undefined) {
if (progress < 30) {
progressValid = false;
}
}
}
}
if(firstSeenValid && progressValid) {
// a prior candidate has been playing for more than 30 seconds and passed progress test, time to check statefuls
const matchingRecent = this.existingDiscovered(candidate); //sRecentlyPlayed.find(x => playObjDataMatch(x, candidate));
let stPrefix = `[Platform ${groupId}] (Stateful Play) ${buildTrackString(candidate, {include: ['trackId', 'artist', 'track']})}`;
if(matchingRecent === undefined) {
this.logger.debug(`${stPrefix} added after being seen for 30 seconds and not matching any prior plays`);
newStatefulPlays.push(candidate);
//sRecentlyPlayed.push(candidate);
} else {
const {data: { playDate, duration }} = candidate;
const {data: { playDate: rplayDate }} = matchingRecent;
if(!playDate.isSame(rplayDate)) {
if(duration !== undefined) {
if(playDate.isAfter(rplayDate.add(duration, 's'))) {
this.logger.debug(`${stPrefix} added after being seen for 30 seconds and having a different timestamp than a prior play`);
newStatefulPlays.push(candidate);
//sRecentlyPlayed.push(candidate);
}
} else {
const discoveredPlays = this.getRecentlyDiscoveredPlaysByPlatform(candidate);
if(discoveredPlays.length === 0 || !playObjDataMatch(discoveredPlays[0], candidate)) {
// if most recent stateful play is not this track we'll add it
this.logger.debug(`${stPrefix} added after being seen for 30 seconds. Matched other recent play but could not determine time frame due to missing duration. Allowed due to not being last played track.`);
newStatefulPlays.push(candidate);
}
//sRecentlyPlayed.push(candidate);
}
} else if(!playObjDataMatch(this.statefulRecentlyPlayed[0], candidate)) {
// if most recent stateful play is not this track we'll add it
this.logger.debug(`${stPrefix} added after being seen for 30 seconds. Matched other recent play but could not determine time frame due to missing duration. Allowed due to not being last played track.`);
newStatefulPlays.push(candidate);
this.statefulRecentlyPlayed.push(candidate);
}
}
}
}
//sRecentlyPlayed.sort(sortByPlayDate);
//this.statefulRecentlyPlayed.set(groupId, sRecentlyPlayed);
}
this.statefulRecentlyPlayed.sort(sortByPlayDate);
}
return newStatefulPlays;
//return this.getFlatStatefulRecentlyPlayed();
//return newStatefulPlays;
}
recentlyPlayedTrackIsValid = (playObj: any) => {
+204
View File
@@ -0,0 +1,204 @@
import MemorySource from "./MemorySource.js";
import {MopidySourceConfig} from "../common/infrastructure/config/source/mopidy.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import dayjs from "dayjs";
import Mopidy, {models} from "mopidy";
import {URL} from "url";
import normalizeUrl from 'normalize-url';
import {EventEmitter} from "events";
import pEvent from 'p-event';
import winston from 'winston';
import {RecentlyPlayedOptions} from "./AbstractSource.js";
import request from "superagent";
import {buildTrackString, removeDuplicates} from "../utils.js";
import {SubsonicSource} from "./SubsonicSource.js";
export class MopidySource extends MemorySource {
declare config: MopidySourceConfig;
albumBlacklist: string[] = [];
uriWhitelist: string[] = [];
uriBlacklist: string[] = [];
url: URL;
client: Mopidy;
clientReady: boolean = false;
constructor(name: any, config: MopidySourceConfig, internal: InternalConfig, emitter: EventEmitter) {
const {
data = {}
} = config;
const {
albumBlacklist = ['Soundcloud'],
uriWhitelist = [],
uriBlacklist = [],
interval = 10,
maxInterval = 30,
...rest
} = data;
super('mopidy', name, {...config, data: {interval, maxInterval, ...rest}}, internal, emitter);
this.albumBlacklist = albumBlacklist.map(x => x.toLocaleLowerCase());
this.uriWhitelist = uriWhitelist.map(x => x.toLocaleLowerCase());
this.uriBlacklist = uriBlacklist.map(x => x.toLocaleLowerCase());
const {
data: {
url = 'ws://localhost:6680/mopidy/ws/'
} = {}
} = config;
this.url = MopidySource.parseConnectionUrl(url);
this.client = new Mopidy({
autoConnect: false,
webSocketUrl: this.url.toString(),
// @ts-ignore
console: winston.loggers.get('noop')
});
this.client.on('state:offline', () => {
this.logger.verbose('Lost connection to server');
this.clientReady = false;
});
this.client.on('state:online', () => {
this.logger.verbose('Connected to server');
this.clientReady = true;
});
this.client.on('reconnecting', () => {
this.logger.verbose('Retrying connection to server...');
});
this.canPoll = true;
}
static parseConnectionUrl(val: string) {
const normal = normalizeUrl(val, {removeTrailingSlash: false, normalizeProtocol: true})
const url = new URL(normal);
// default WS
if (url.protocol === 'http:') {
url.protocol = 'ws';
} else if (url.protocol === 'https:') {
url.protocol = 'wss';
}
if (url.port === null || url.port === '') {
url.port = '6680';
}
if (url.pathname === '/') {
url.pathname = '/mopidy/ws/';
} else if (url.pathname === '/mopidy/ws') {
url.pathname = '/mopidy/ws/';
}
return url;
}
initialize = async () => {
const {
data: {
url
} = {}
} = this.config;
this.logger.debug(`Config URL: '${url ?? '(None Given)'}' => Normalized: '${this.url.toString()}'`)
this.client.connect();
const res = await Promise.race([
pEvent(this.client, 'state:online'),
pEvent(this.client, 'websocket:error'),
pEvent(this.client, 'websocket:close'),
]);
if (res === undefined) {
this.logger.info('Connection OK');
this.initialized = true;
return true;
} else {
this.logger.error(`Could not connect. Error => ${(res as Error).message}`);
this.client.close();
return false;
}
}
formatPlayObj(obj: models.Track, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = true, trackProgressPosition = undefined} = options;
const {
artists: artistsVal,
album: albumVal,
name,
uri, // like 'local:track...' 'soundcloud:song...'
length: lengthVal,
composers = [],
performers = []
} = obj;
let artists = artistsVal === null ? [] : artistsVal;
let album: models.Album = albumVal === null ? {} as models.Album : albumVal;
if (this.albumBlacklist.length > 0 && album.name !== undefined && this.albumBlacklist.some(x => album.name.toLocaleLowerCase().includes(x))) {
album = {} as models.Album;
}
const length = lengthVal === null ? undefined : lengthVal;
const {
name: albumName,
artists: albumArtists = []
} = album as models.Album;
if ((artists.length === 0 || artists.every(x => x.name.toLocaleLowerCase().includes('various'))) && albumArtists.length > 0) {
artists = albumArtists;
}
if (artists.length === 0 && composers.length > 0) {
artists = composers;
}
if (artists.length === 0 && performers.length > 0) {
artists = performers;
}
return {
data: {
track: name,
album: albumName,
artists: artists.length > 0 ? artists.map(x => x.name) : [],
duration: Math.round(length / 1000),
playDate: dayjs()
},
meta: {
source: 'mopidy',
trackId: uri,
newFromSource,
trackProgressPosition: trackProgressPosition !== undefined ? Math.round(trackProgressPosition / 1000) : undefined,
deviceId: name,
}
}
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
if (!this.clientReady) {
this.logger.warn('Cannot actively poll since client is not connected.');
return [];
}
const currTrack = await this.client.playback.getCurrentTrack();
const playback = await this.client.playback.getTimePosition();
let play: PlayObject | undefined = currTrack === null ? undefined : this.formatPlayObj(currTrack, {trackProgressPosition: playback});
if(play !== undefined) {
if (this.uriWhitelist.length > 0) {
const match = this.uriWhitelist.find(x => currTrack.uri.includes(x));
if (match === undefined) {
this.logger.debug(`URI for currently playing (${currTrack.uri}) did not match any in whitelist. Will not track play ${buildTrackString(play)}`);
play = undefined;
}
} else if (this.uriBlacklist.length > 0) {
const match = this.uriWhitelist.find(x => currTrack.uri.includes(x));
if (match !== undefined) {
this.logger.debug(`URI for currently playing (${currTrack.uri}) matched from blacklist (${match}). Will not track play ${buildTrackString(play)}`);
play = undefined;
}
}
}
return this.processRecentPlays(play === undefined ? [] : [play]);
}
}
+25 -11
View File
@@ -1,20 +1,31 @@
import dayjs from "dayjs";
import {buildTrackString, createLabelledLogger} from "../utils.js";
import {
buildTrackString,
combinePartsToString,
mergeArr,
truncateStringToLength
} from "../utils.js";
import AbstractSource from "./AbstractSource.js";
import formidable from 'formidable';
import concatStream from 'concat-stream';
import {PlexSourceConfig} from "../common/infrastructure/config/source/plex.js";
import {InternalConfig, PlayObject, SourceType} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject, SourceType} from "../common/infrastructure/Atomic.js";
import EventEmitter from "events";
import winston from "winston";
const shortDeviceId = truncateStringToLength(10, '');
export default class PlexSource extends AbstractSource {
users: string[];
libraries: string[];
servers: string[];
multiPlatform: boolean = true;
declare config: PlexSourceConfig;
constructor(name: any, config: PlexSourceConfig, internal: InternalConfig, type: SourceType = 'plex') {
super(type, name, config, internal);
constructor(name: any, config: PlexSourceConfig, internal: InternalConfig, type: SourceType = 'plex',emitter: EventEmitter) {
super(type, name, config, internal, emitter);
const {data: {user = [], libraries = [], servers = []} = {}} = config
if (!Array.isArray(user)) {
@@ -46,7 +57,8 @@ export default class PlexSource extends AbstractSource {
this.initialized = true;
}
static formatPlayObj(obj: any, newFromSource = false): PlayObject {
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
event,
Account: {
@@ -69,6 +81,10 @@ export default class PlexSource extends AbstractSource {
// @ts-expect-error TS(2525): Initializer provides no value for this binding ele... Remove this comment to see the full error message
title: server
} = {},
Player: {
title,
uuid,
}
} = obj;
return {
data: {
@@ -85,6 +101,7 @@ export default class PlexSource extends AbstractSource {
server,
source: 'Plex',
newFromSource,
deviceId: combinePartsToString([shortDeviceId(uuid), title])
}
}
}
@@ -157,16 +174,13 @@ export default class PlexSource extends AbstractSource {
return true;
}
handle = async (playObj: any, allClients: any) => {
handle = async (playObj: any) => {
if (!this.isValidEvent(playObj)) {
return;
}
this.logger.info(`New Track => ${buildTrackString(playObj)}`);
try {
await allClients.scrobble(playObj, {scrobbleTo: this.clients, scrobbleFrom: this.identifier});
// only gets hit if we scrobbled ok
this.tracksDiscovered++;
this.scrobble([playObj]);
} catch (e) {
this.logger.error('Encountered error while scrobbling')
this.logger.error(e)
@@ -176,7 +190,7 @@ export default class PlexSource extends AbstractSource {
export const plexRequestMiddle = () => {
const plexLog = createLabelledLogger('plexReq', 'Plex Request');
const plexLog = winston.loggers.get('app').child({labels: ['Plex Request']}, mergeArr);
return async (req: any, res: any, next: any) => {
+154 -26
View File
@@ -1,4 +1,4 @@
import {createLabelledLogger, readJson, validateJson} from "../utils.js";
import {mergeArr, parseBool, readJson, validateJson} from "../utils.js";
import SpotifySource from "./SpotifySource.js";
import PlexSource from "./PlexSource.js";
import TautulliSource from "./TautulliSource.js";
@@ -8,7 +8,7 @@ import LastfmSource from "./LastfmSource.js";
import DeezerSource from "./DeezerSource.js";
import {ConfigMeta, InternalConfig, SourceType, sourceTypes} from "../common/infrastructure/Atomic.js";
import {configDir as defaultConfigDir} from "../common/index.js";
import {Logger} from "winston";
import winston, {Logger} from "winston";
import {SourceAIOConfig, SourceConfig} from "../common/infrastructure/config/source/sources.js";
import {DeezerData, DeezerSourceConfig} from "../common/infrastructure/config/source/deezer.js";
import {LastfmClientConfig} from "../common/infrastructure/config/client/lastfm.js";
@@ -22,6 +22,19 @@ import {AIOConfig} from "../common/infrastructure/config/aioConfig.js";
import * as aioSchema from "../common/schema/aio-source.json";
import * as sourceSchema from "../common/schema/source.json";
import {LastfmSourceConfig} from "../common/infrastructure/config/source/lastfm.js";
import YTMusicSource from "./YTMusicSource.js";
import {YTMusicSourceConfig} from "../common/infrastructure/config/source/ytmusic.js";
import {MPRISData, MPRISSourceConfig} from "../common/infrastructure/config/source/mpris.js";
import {MPRISSource} from "./MPRISSource.js";
import EventEmitter from "events";
import {MopidySource} from "./MopidySource.js";
import {MopidySourceConfig} from "../common/infrastructure/config/source/mopidy.js";
import ListenbrainzSource from "./ListenbrainzSource.js";
import {ListenBrainzSourceConfig} from "../common/infrastructure/config/source/listenbrainz.js";
import {JRiverSource} from "./JRiverSource.js";
import {JRiverData, JRiverSourceConfig} from "../common/infrastructure/config/source/jriver.js";
import {KodiSource} from "./KodiSource.js";
import {KodiData, KodiSourceConfig} from "../common/infrastructure/config/source/kodi.js";
type groupedNamedConfigs = {[key: string]: ParsedConfig[]};
@@ -34,10 +47,13 @@ export default class ScrobbleSources {
configDir: string;
localUrl: string;
constructor(localUrl: string, configDir: string = defaultConfigDir) {
emitter: EventEmitter;
constructor(emitter: EventEmitter, localUrl: string, configDir: string = defaultConfigDir) {
this.emitter = emitter;
this.configDir = configDir;
this.localUrl = localUrl;
this.logger = createLabelledLogger('sources', 'Sources');
this.logger = winston.loggers.get('app').child({labels: ['Sources']}, mergeArr);
}
getByName = (name: any) => {
@@ -52,6 +68,33 @@ export default class ScrobbleSources {
return this.sources.find(x => x.name === name && x.type === type);
}
async getStatusSummary(type?: string, name?: string): Promise<[boolean, string[]]> {
let sources: AbstractSource[]
let sourcesReady = true;
let messages: string[] = [];
if(type !== undefined) {
sources = this.getByType(type);
} else if(name !== undefined) {
sources = [this.getByName(name)];
} else {
sources = this.sources;
}
for(const source of sources) {
if(source.requiresAuth && !source.authed) {
sourcesReady = false;
messages.push(`Source ${source.type} - ${source.name} requires authentication.`);
}
if(source.canPoll && !source.polling) {
sourcesReady = false;
messages.push(`Source ${source.type} - ${source.name} is not polling.`);
}
}
return [sourcesReady, messages];
}
buildSourcesFromConfig = async (additionalConfigs: ParsedConfig[] = []) => {
let configs: ParsedConfig[] = additionalConfigs;
@@ -199,6 +242,61 @@ export default class ScrobbleSources {
});
}
break;
case 'mpris':
const shouldUse = parseBool(process.env.MPRIS_ENABLE);
const mp = {
blacklist: process.env.MPRIS_BLACKLIST,
whitelist: process.env.MPRIS_WHITELIST
}
if (!Object.values(mp).every(x => x === undefined) || shouldUse) {
configs.push({
type: 'mpris',
name: 'unnamed',
source: 'ENV',
mode: 'single',
configureAs: defaultConfigureAs,
data: mp as MPRISData
});
}
break;
case 'listenbrainz':
// sane default for lastfm is that user want to scrobble TO it, not FROM it -- this is also existing behavior
defaultConfigureAs = 'client';
break;
case 'jriver':
const jr = {
url: process.env.JRIVER_URL,
username: process.env.JRIVER_USER,
password: process.env.JRIVER_PASSWORD
}
if (!Object.values(jr).every(x => x === undefined)) {
configs.push({
type: 'jriver',
name: 'unnamed',
source: 'ENV',
mode: 'single',
configureAs: defaultConfigureAs,
data: jr as JRiverData
});
}
break;
case 'kodi':
const ko = {
url: process.env.KODI_URL,
username: process.env.KODI_USER,
password: process.env.KODI_PASSWORD
}
if (!Object.values(ko).every(x => x === undefined)) {
configs.push({
type: 'kodi',
name: 'unnamed',
source: 'ENV',
mode: 'single',
configureAs: defaultConfigureAs,
data: ko as KodiData
});
}
break;
default:
break;
}
@@ -226,14 +324,25 @@ export default class ScrobbleSources {
try {
const validConfig = validateJson<SourceConfig>(rawConf, sourceSchema, this.logger);
if(sourceType !== 'lastfm' || ((validConfig as LastfmSourceConfig).configureAs === 'source')) {
// @ts-ignore
const parsedConfig: ParsedConfig = {
...rawConf,
source: `${sourceType}.json`,
type: sourceType
}
// @ts-ignore
const parsedConfig: ParsedConfig = {
...rawConf,
source: `${sourceType}.json`,
type: sourceType
}
if(!['lastfm','listenbrainz'].includes(sourceType) || ((validConfig as LastfmSourceConfig | ListenBrainzSourceConfig).configureAs === 'source')) {
configs.push(parsedConfig);
} else {
if('configureAs' in validConfig) {
if(validConfig.configureAs === 'source') {
configs.push(parsedConfig);
} else {
this.logger.debug(`${sourceType} has 'configureAs: client' so will skip adding as a source`);
}
} else {
this.logger.debug(`${sourceType} did not have 'configureAs' specified! Assuming 'client' so will skip adding as a source`);
}
}
} catch (e: any) {
this.logger.error(`The config entry at index ${i} from ${sourceType}.json was not valid`);
@@ -305,7 +414,8 @@ export default class ScrobbleSources {
const internal: InternalConfig = {
localUrl: this.localUrl,
configDir: this.configDir
configDir: this.configDir,
logger: this.logger
};
const {type, name, data: d = {}} = clientConfig;
@@ -318,25 +428,43 @@ export default class ScrobbleSources {
let newSource;
switch (type) {
case 'spotify':
newSource = new SpotifySource(name, compositeConfig as SpotifySourceConfig, internal);
newSource = new SpotifySource(name, compositeConfig as SpotifySourceConfig, internal, this.emitter);
break;
case 'plex':
newSource = await new PlexSource(name, compositeConfig as PlexSourceConfig, internal);
newSource = await new PlexSource(name, compositeConfig as PlexSourceConfig, internal, 'plex', this.emitter);
break;
case 'tautulli':
newSource = await new TautulliSource(name, compositeConfig as TautulliSourceConfig, internal);
newSource = await new TautulliSource(name, compositeConfig as TautulliSourceConfig, internal, this.emitter);
break;
case 'subsonic':
newSource = new SubsonicSource(name, compositeConfig as SubSonicSourceConfig, internal);
newSource = new SubsonicSource(name, compositeConfig as SubSonicSourceConfig, internal, this.emitter);
break;
case 'jellyfin':
newSource = await new JellyfinSource(name, compositeConfig as JellySourceConfig, internal);
newSource = await new JellyfinSource(name, compositeConfig as JellySourceConfig, internal, this.emitter);
break;
case 'lastfm':
newSource = await new LastfmSource(name, compositeConfig as LastfmClientConfig, internal);
newSource = await new LastfmSource(name, compositeConfig as LastfmClientConfig, internal, this.emitter);
break;
case 'deezer':
newSource = await new DeezerSource(name, compositeConfig as DeezerSourceConfig, internal);
newSource = await new DeezerSource(name, compositeConfig as DeezerSourceConfig, internal, this.emitter);
break;
case 'ytmusic':
newSource = await new YTMusicSource(name, compositeConfig as YTMusicSourceConfig, internal, this.emitter);
break;
case 'mpris':
newSource = await new MPRISSource(name, compositeConfig as MPRISSourceConfig, internal, this.emitter);
break;
case 'mopidy':
newSource = await new MopidySource(name, compositeConfig as MopidySourceConfig, internal, this.emitter);
break;
case 'listenbrainz':
newSource = await new ListenbrainzSource(name, compositeConfig as ListenBrainzSourceConfig, internal, this.emitter);
break;
case 'jriver':
newSource = await new JRiverSource(name, compositeConfig as JRiverSourceConfig, internal, this.emitter);
break;
case 'kodi':
newSource = await new KodiSource(name, compositeConfig as KodiSourceConfig, internal, this.emitter);
break;
default:
break;
@@ -347,19 +475,19 @@ export default class ScrobbleSources {
throw new Error(`Source of type ${type} was not recognized??`);
}
if(newSource.initialized === false) {
this.logger.debug(`(${name}) Attempting ${type} initialization...`);
this.logger.debug(`Attempting ${type} (${name}) initialization...`);
if ((await newSource.initialize()) === false) {
this.logger.error(`(${name}) ${type} source failed to initialize. Source needs to be successfully initialized before activity capture can begin.`);
this.logger.error(`${type} (${name}) source failed to initialize. Source needs to be successfully initialized before activity capture can begin.`);
return;
} else {
this.logger.info(`(${name}) ${type} source initialized`);
this.logger.info(`${type} (${name}) source initialized`);
}
} else {
this.logger.info(`(${name}) ${type} source initialized`);
this.logger.info(`${type} (${name}) source initialized`);
}
if(newSource.requiresAuth && !newSource.authed) {
this.logger.debug(`(${name}) Checking ${type} source auth...`);
this.logger.debug(`Checking ${type} (${name}) source auth...`);
let success;
try {
success = await newSource.testAuth();
@@ -367,9 +495,9 @@ export default class ScrobbleSources {
success = false;
}
if(!success) {
this.logger.warn(`(${name}) ${type} source auth failed.`);
this.logger.warn(`${type} (${name}) source auth failed.`);
} else {
this.logger.info(`(${name}) ${type} source auth OK`);
this.logger.info(`${type} (${name}) source auth OK`);
}
}
+185 -31
View File
@@ -2,18 +2,28 @@ import dayjs from "dayjs";
import {
readJson,
writeFile,
sortByPlayDate, sleep, parseRetryAfterSecsFromObj,
sortByOldestPlayDate, sleep, parseRetryAfterSecsFromObj, buildTrackString, truncateStringToLength, combinePartsToString,
} from "../utils.js";
import SpotifyWebApi from "spotify-web-api-node";
import AbstractSource, {RecentlyPlayedOptions} from "./AbstractSource.js";
import {SpotifySourceConfig} from "../common/infrastructure/config/source/spotify.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import PlayHistoryObject = SpotifyApi.PlayHistoryObject;
import EventEmitter from "events";
import CurrentlyPlayingObject = SpotifyApi.CurrentlyPlayingObject;
import TrackObjectFull = SpotifyApi.TrackObjectFull;
import ArtistObjectSimplified = SpotifyApi.ArtistObjectSimplified;
import AlbumObjectSimplified = SpotifyApi.AlbumObjectSimplified;
import UserDevice = SpotifyApi.UserDevice;
import MemorySource from "./MemorySource.js";
import {ErrorWithCause} from "pony-cause";
const scopes = ['user-read-recently-played', 'user-read-currently-playing'];
const scopes = ['user-read-recently-played', 'user-read-currently-playing', 'user-read-playback-state', 'user-read-playback-position'];
const state = 'random';
export default class SpotifySource extends AbstractSource {
const shortDeviceId = truncateStringToLength(10, '');
export default class SpotifySource extends MemorySource {
spotifyApi: SpotifyWebApi;
workingCredsPath: string;
@@ -21,18 +31,20 @@ export default class SpotifySource extends AbstractSource {
requiresAuth = true;
requiresAuthInteraction = true;
canGetState = false;
declare config: SpotifySourceConfig;
constructor(name: any, config: SpotifySourceConfig, internal: InternalConfig) {
super('spotify', name, config, internal);
constructor(name: any, config: SpotifySourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('spotify', name, config, internal, emitter);
const {
data: {
interval = 60,
interval = 30,
} = {}
} = config;
if (interval < 15) {
this.logger.warn('Interval should be above 30 seconds...😬');
this.logger.warn('Interval should be 15 seconds or above...😬');
}
this.config.data.interval = interval;
@@ -41,23 +53,85 @@ export default class SpotifySource extends AbstractSource {
this.canPoll = true;
}
static formatPlayObj(obj: PlayHistoryObject, newFromSource = false): PlayObject {
static formatPlayObj(obj: object, options: FormatPlayObjectOptions = {}): PlayObject {
const {
track: {
artists = [],
name,
id,
duration_ms,
album: {
name: albumName,
} = {},
newFromSource = false
} = options;
let artists: ArtistObjectSimplified[];
let album: AlbumObjectSimplified;
let name: string;
let duration_ms: number;
let played_at: string;
let id: string;
let url: string;
let playbackPosition: number | undefined;
let deviceId: string | undefined;
if (asPlayHistoryObject(obj)) {
const {
track,
played_at: pa
} = obj;
const {
artists: art = [],
name: n,
id: i,
duration_ms: dm,
album: a,
external_urls: {
spotify,
} = {}
} = {},
played_at
} = obj;
//let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(',');
} = track;
played_at = pa;
artists = art;
name = n;
id = i;
duration_ms = dm;
album = a;
url = spotify;
} else if (asCurrentlyPlayingObject(obj)) {
const {
is_playing,
progress_ms,
timestamp,
device: {
id: deviceIdentifier,
name: deviceName
},
item,
} = obj;
const {
artists: art,
name: n,
id: i,
duration_ms: dm,
album: a,
external_urls: {
spotify,
} = {}
} = item as TrackObjectFull;
played_at = dayjs(timestamp).toISOString();
artists = art;
name = n;
id = i;
duration_ms = dm;
album = a;
url = spotify;
playbackPosition = progress_ms / 1000;
deviceId = combinePartsToString([shortDeviceId(deviceIdentifier), deviceName]);
} else {
throw new Error('Could not determine format of spotify response data');
}
const {name: albumName} = album || {};
return {
data: {
artists: artists.map((x: any) => x.name),
@@ -67,12 +141,13 @@ export default class SpotifySource extends AbstractSource {
playDate: dayjs(played_at),
},
meta: {
trackLength: duration_ms / 1000,
deviceId,
source: 'Spotify',
sourceId: id,
trackId: id,
trackProgressPosition: playbackPosition,
newFromSource,
url: {
web: spotify
web: url
}
}
};
@@ -144,7 +219,7 @@ export default class SpotifySource extends AbstractSource {
await this.callApi<ReturnType<typeof this.spotifyApi.getMe>>(((api: any) => api.getMe()));
this.authed = true;
} catch (e) {
this.logger.error('Could not successfully communicate with Spotify API');
this.logger.error(new ErrorWithCause('Could not successfully communicate with Spotify API', {cause: e}));
this.authed = false;
}
return this.authed;
@@ -176,14 +251,70 @@ export default class SpotifySource extends AbstractSource {
}
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
const plays: PlayObject[] = [];
if(this.canGetState) {
const state = await this.getCurrentPlaybackState();
if(state.play !== undefined) {
if(state.device.is_private_session) {
this.logger.debug(`Will not track play on Device ${state.device.name} because it is a private session: ${buildTrackString(state.play)}`);
} else {
plays.push(state.play);
}
}
} else {
const currPlay = await this.getNowPlaying();
if(currPlay !== undefined) {
plays.push(currPlay);
}
}
return this.processRecentPlays(plays, true);
}
getPlayHistory = async (options: RecentlyPlayedOptions = {}) => {
const {limit = 20} = options;
const func = (api: SpotifyWebApi) => api.getMyRecentlyPlayedTracks({
limit
});
const result = await this.callApi<ReturnType<typeof this.spotifyApi.getMyRecentlyPlayedTracks>>(func);
return result.body.items.map((x: any) => SpotifySource.formatPlayObj(x)).sort(sortByPlayDate);
return result.body.items.map((x: any) => SpotifySource.formatPlayObj(x)).sort(sortByOldestPlayDate);
}
getNowPlaying = async () => {
const func = (api: SpotifyWebApi) => api.getMyCurrentPlayingTrack();
const playingRes = await this.callApi<ReturnType<typeof this.spotifyApi.getMyCurrentPlayingTrack>>(func);
const {body: {item}} = playingRes;
if(item !== undefined && item !== null) {
return SpotifySource.formatPlayObj(playingRes.body, {newFromSource: true});
}
return undefined;
}
getCurrentPlaybackState = async (logError = true): Promise<{device?: UserDevice, play?: PlayObject}> => {
try {
const funcState = (api: SpotifyWebApi) => api.getMyCurrentPlaybackState();
const res = await this.callApi<ReturnType<typeof this.spotifyApi.getMyCurrentPlaybackState>>(funcState);
const {body: {
device,
item
} = {}} = res;
return {
device: device === null ? undefined : device,
play: item !== null && item !== undefined ? SpotifySource.formatPlayObj(res.body, {newFromSource: true}) : undefined
}
} catch (e) {
if(logError) {
this.logger.error(`Error occurred while trying to retrieve current playback state: ${e.message}`);
}
throw e;
}
}
/* getDevices = async () => {
const funcDevice = (api: SpotifyWebApi) => api.getMyDevices();
return await this.callApi<ReturnType<typeof this.spotifyApi.getMyDevices>>(funcDevice);
}*/
callApi = async <T>(func: (api: SpotifyWebApi) => Promise<any>, retries = 0): Promise<T> => {
const {
maxRequestRetries = 1,
@@ -216,7 +347,7 @@ export default class SpotifySource extends AbstractSource {
return await func(this.spotifyApi);
} catch (ee) {
this.logger.error('Refreshing access token encountered an error');
this.logger.error(ee, {label: 'Spotify'});
this.logger.error(ee);
throw ee;
}
} else if(maxRequestRetries > retries) {
@@ -226,17 +357,40 @@ export default class SpotifySource extends AbstractSource {
return this.callApi(func, retries + 1);
} else {
this.logger.error(`Request failed on retry (${retries}) with no more retries permitted (max ${maxRequestRetries})`);
this.logger.error(e, {label: 'Spotify'});
throw e;
const error = new ErrorWithCause(`Request failed on retry (${retries}) with no more retries permitted (max ${maxRequestRetries})`, {cause: e});
this.logger.error(error);
throw error;
}
}
}
poll = async (allClients: any) => {
poll = async () => {
if (this.spotifyApi === undefined) {
this.logger.warn('Cannot poll spotify without valid credentials configuration')
return;
}
await this.startPolling(allClients);
// test capabilities
try {
await this.getCurrentPlaybackState(false);
this.canGetState = true;
} catch (e) {
this.logger.warn('multi-scrobbler does not have sufficient permissions to access Spotify API "Get Playback State". MS will continue to work but accuracy for determining if/when a track played from a Spotify Connect device (smart device controlled through Spotify app) may be degraded. To fix this re-authenticate MS with Spotify and restart polling.');
}
this.logger.info('Checking recently played API for tracks to backlog...');
const backlogPlays = await this.getPlayHistory({formatted: true});
this.scrobble(backlogPlays);
this.logger.info('Backlog complete.');
await this.startPolling();
}
}
const asPlayHistoryObject = (obj: object): obj is PlayHistoryObject => {
return 'played_at' in obj;
}
const asCurrentlyPlayingObject = (obj: object): obj is CurrentlyPlayingObject => {
return 'is_playing' in obj;
}
+19 -13
View File
@@ -2,11 +2,12 @@ import request from 'superagent';
import * as crypto from 'crypto';
import dayjs from "dayjs";
import isSameOrAfter from "dayjs/plugin/isSameOrAfter.js";
import {buildTrackString, parseRetryAfterSecsFromObj, sleep} from "../utils.js";
import {buildTrackString, parseRetryAfterSecsFromObj, removeDuplicates, sleep} from "../utils.js";
import MemorySource from "./MemorySource.js";
import {SubSonicSourceConfig} from "../common/infrastructure/config/source/subsonic.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {RecentlyPlayedOptions} from "./AbstractSource.js";
import EventEmitter from "events";
dayjs.extend(isSameOrAfter);
@@ -14,21 +15,21 @@ export class SubsonicSource extends MemorySource {
requiresAuth = true;
multiPlatform: boolean = true;
declare config: SubSonicSourceConfig;
constructor(name: any, config: SubSonicSourceConfig, internal: InternalConfig) {
constructor(name: any, config: SubSonicSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
// default to quick interval so we can get a decently accurate nowPlaying
const {
data: {
// @ts-ignore
interval = 10,
// @ts-ignore
maxSleep = 30,
maxInterval = 30,
...restData
} = {}
} = config;
const subsonicConfig = {...config, data: {...restData, internal, maxSleep}};
super('subsonic', name, subsonicConfig, internal);
const subsonicConfig = {...config, data: {...restData, internal, maxInterval}};
super('subsonic', name, subsonicConfig, internal,emitter);
const {data: {user, password, url} = {}} = this.config;
@@ -45,7 +46,8 @@ export class SubsonicSource extends MemorySource {
this.canPoll = true;
}
static formatPlayObj(obj: any, newFromSource = false): PlayObject {
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
id,
title,
@@ -53,6 +55,8 @@ export class SubsonicSource extends MemorySource {
artist,
duration, // seconds
minutesAgo,
playerId,
username,
} = obj;
return {
data: {
@@ -65,10 +69,11 @@ export class SubsonicSource extends MemorySource {
playDate: minutesAgo === 0 ? dayjs().startOf('minute') : dayjs().startOf('minute').subtract(minutesAgo, 'minute'),
},
meta: {
trackLength: duration,
source: 'Subsonic',
sourceId: id,
trackId: id,
newFromSource,
user: username,
deviceId: playerId
}
}
}
@@ -186,7 +191,8 @@ export class SubsonicSource extends MemorySource {
entry = []
} = {}
} = resp;
this.processRecentPlays(entry.map((x: any) => formatted ? SubsonicSource.formatPlayObj(x) : x));
return this.statefulRecentlyPlayed;
// sometimes subsonic sources will return the same track as being played twice on the same player, need to remove this so we don't duplicate plays
const deduped = removeDuplicates(entry.map(SubsonicSource.formatPlayObj));
return this.processRecentPlays(deduped);
}
}
+24 -7
View File
@@ -1,16 +1,21 @@
import dayjs from "dayjs";
import PlexSource from "./PlexSource.js";
import {TautulliSourceConfig} from "../common/infrastructure/config/source/tautulli.js";
import {InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {combinePartsToString, truncateStringToLength} from "../utils.js";
import EventEmitter from "events";
const shortDeviceId = truncateStringToLength(10, '');
export default class TautulliSource extends PlexSource {
declare config: TautulliSourceConfig;
constructor(name: any, config: TautulliSourceConfig, internal: InternalConfig) {
super(name, config, internal, 'tautulli');
constructor(name: any, config: TautulliSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super(name, config, internal, 'tautulli', emitter);
}
static formatPlayObj(obj: any, newFromSource = false): PlayObject {
static formatPlayObj(obj: any, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
artist_name,
track_name,
@@ -23,12 +28,23 @@ export default class TautulliSource extends PlexSource {
version,
duration,
username,
library
} = obj;
library,
machine_id = '',
session_key,
action,
platform,
device,
player,
} = obj.body;
let artists = [artist_name];
if (track_artist !== undefined && track_artist !== artist_name) {
artists.push(track_artist);
}
if(action === undefined) {
//TODO why does TS think logger doesn't exist?
// @ts-ignore
this.logger.warn(`Payload did contain property 'action', assuming it should be 'watched'`);
}
return {
data: {
artists,
@@ -39,14 +55,15 @@ export default class TautulliSource extends PlexSource {
},
meta: {
title,
event: action === 'watched' ? 'media.scrobble' : action,
library: library_name ?? library,
server,
sourceVersion: version,
mediaType: media_type,
user: username,
trackLength: duration,
source: 'Tautulli',
newFromSource,
deviceId: combinePartsToString([shortDeviceId(machine_id), session_key, player])
}
}
}
+199
View File
@@ -0,0 +1,199 @@
import YouTubeMusic from "youtube-music-ts-api";
import AbstractSource, {RecentlyPlayedOptions} from "./AbstractSource.js";
import {FormatPlayObjectOptions, InternalConfig, PlayObject} from "../common/infrastructure/Atomic.js";
import {IYouTubeMusicAuthenticated} from "youtube-music-ts-api/interfaces-primary";
import dayjs from "dayjs";
import {parseDurationFromTimestamp, playObjDataMatch} from "../utils.js";
import {IPlaylistDetail, ITrackDetail} from "youtube-music-ts-api/interfaces-supplementary";
import {YTMusicSourceConfig} from "../common/infrastructure/config/source/ytmusic.js";
import EventEmitter from "events";
export default class YTMusicSource extends AbstractSource {
apiInstance?: IYouTubeMusicAuthenticated
requiresAuth = true;
declare config: YTMusicSourceConfig
recentlyPlayed: PlayObject[] = [];
constructor(name: string, config: YTMusicSourceConfig, internal: InternalConfig, emitter: EventEmitter) {
super('ytmusic', name, config, internal, emitter);
this.canPoll = true;
}
static formatPlayObj(obj: ITrackDetail, options: FormatPlayObjectOptions = {}): PlayObject {
const {newFromSource = false} = options;
const {
id,
title,
album: albumData,
artists: artistsData,
duration: durTimestamp, // string timestamp
} = obj;
let artists = undefined,
album = undefined,
duration = undefined;
if(artistsData !== undefined) {
artists = artistsData.map(x => x.name) as string[];
}
if(albumData !== undefined) {
album = albumData.name;
}
if(durTimestamp !== undefined) {
const durObj = parseDurationFromTimestamp(durTimestamp);
duration = durObj.asSeconds();
}
return {
data: {
artists,
album,
track: title,
duration,
// if object is new from source then we know we've picked it up AFTER we started polling so it can't be older than 1 minute (default polling interval)
playDate: newFromSource ? dayjs().startOf('minute') : undefined,
},
meta: {
source: 'YTMusic',
trackId: id,
newFromSource,
}
}
}
recentlyPlayedTrackIsValid = (playObj: PlayObject) => {
return playObj.meta.newFromSource;
}
api = async (): Promise<IYouTubeMusicAuthenticated> => {
if(this.apiInstance !== undefined) {
return this.apiInstance;
}
const ytm = new YouTubeMusic();
try {
this.apiInstance = await ytm.authenticate(this.config.data.cookie, this.config.data.authUser);
} catch (e: any) {
this.logger.error('Failed to authenticate', e);
throw e;
}
return this.apiInstance;
}
protected getLibraryHistory = async (): Promise<IPlaylistDetail> => {
// internally for this call YT returns a *list* of playlists with decreasing granularity from most recent to least recent like this:
// * Today
// * Yesterday
// * ....
// * January 2023
//
// the playlist returned can therefore change abruptly IE MS started yesterday and new music listened to today -> "today" playlist is cleared
try {
const playlist = await (await this.api()).getLibraryHistory();
return {tracks: [], ...playlist};
} catch (e) {
throw e;
}
}
/**
* Get the last 20 recently played tracks
* */
getRecentlyPlayed = async (options: RecentlyPlayedOptions = {}) => {
const { display = false } = options;
let playlistDetail: IPlaylistDetail;
try {
playlistDetail = await this.getLibraryHistory();
} catch (e) {
throw e;
}
let newPlays: PlayObject[] = [];
const plays = playlistDetail.tracks.map((x) => YTMusicSource.formatPlayObj(x, {newFromSource: false})).slice(0, 20);
if(this.polling === false) {
this.recentlyPlayed = plays;
newPlays = plays;
} else {
// iterate through each play until we find one that matched the "newest" from the recently played
for (const [i, value] of plays.entries()) {
if (this.recentlyPlayed.length === 0) {
// playlist was empty when we started, nothing to compare to so all tracks are new
newPlays.push(value);
} else if (this.recentlyPlayed.length !== plays.length) { // if there is a difference in list length we need to check for consecutive repeat tracks as well
const match = playObjDataMatch(value, this.recentlyPlayed[0]);
if (!match) {
newPlays.push(value)
} else if (match && plays.length > i + 1 && playObjDataMatch(plays[i + 1], this.recentlyPlayed[0])) { // if it matches but next ALSO matches the current it's a repeat "new"
// check if repeated track
newPlays.push(value)
} else {
break;
}
} else if (!playObjDataMatch(value, this.recentlyPlayed[0])) {
// if the newest doesn't match a play then the play is new
newPlays.push(value);
} else {
// otherwise we're back known plays
break;
}
}
if(newPlays.length > 0) {
newPlays = newPlays.map((x) => {
return {
data: {
...x.data,
playDate: dayjs().startOf('minute')
},
meta: {
...x.meta,
newFromSource: true
}
}
});
this.recentlyPlayed = newPlays.concat(this.recentlyPlayed).slice(0, 20);
}
}
return newPlays;
}
testAuth = async () => {
try {
await this.getRecentlyPlayed();
this.authed = true;
} catch (e) {
if(e.message.includes('Status code: 401')) {
let hint = 'Verify your cookie and authUser are correct.';
if(this.config.data.authUser === undefined) {
hint = `${hint} TIP: 'authUser' is not defined your credentials. If you are using Chrome to retrieve credentials from music.youtube.com make sure the value from the 'X-Goog-AuthUser' is used as 'authUser'.`;
}
this.logger.error(`Authentication failed with the given credentials. ${hint} | Error => ${e.message}`);
}
this.authed = false;
}
return this.authed;
}
poll = async () => {
if(this.authed && !this.polling) {
this.logger.verbose('Hydrating initial recently played tracks for reference.');
const referencePlays = await this.getRecentlyPlayed();
const reversedPlays = [...referencePlays];
// actual order they were discovered in (oldest to newest)
reversedPlays.reverse();
if(this.getFlatRecentlyDiscoveredPlays().length === 0) {
// and add to discovered since its empty
for(const refPlay of reversedPlays) {
this.addPlayToDiscovered(refPlay);
}
}
}
await this.startPolling();
}
}
@@ -1,5 +1,5 @@
import {Logger} from "winston";
import {createLabelledLogger, remoteHostIdentifiers, remoteHostStr} from "../../utils.js";
import winston, {Logger} from "winston";
import {mergeArr, remoteHostIdentifiers, remoteHostStr} from "../../utils.js";
import {Request} from "express";
import {RemoteIdentityParts} from "../../common/infrastructure/Atomic.js";
@@ -13,8 +13,7 @@ export abstract class IngressNotifier {
protected constructor(name: string) {
this.identifier = name;
const label = `Ingress - ${name}`;
this.logger = createLabelledLogger(label, label);
this.logger = winston.loggers.get('app').child({labels: ['Ingress', name]}, mergeArr);
}
public trackIngress(req: Request, isRaw: boolean) {
@@ -1,6 +1,7 @@
import {IngressNotifier} from "./IngressNotifier.js";
import {Request} from "express";
import JellyfinSource from "../JellyfinSource.js";
import {remoteHostIdentifiers, remoteHostStr} from "../../utils.js";
export class JellyfinNotifier extends IngressNotifier {
@@ -11,15 +12,37 @@ export class JellyfinNotifier extends IngressNotifier {
seenServers: Record<string, string> = {};
notifyBySource(req: Request, isRaw: boolean): [boolean, (string | undefined)] {
if(!isRaw) {
const playObj = JellyfinSource.formatPlayObj(req.body);
const parts = remoteHostIdentifiers(req);
let serverIdentifier = `${parts.host}-${parts.proxy ?? ''}`;
if(playObj.meta.server === undefined) {
return [false, `Payload from Jellyfin did not contain server info! Make sure 'Send all Properties' is checked in webhook`];
const playObj = JellyfinSource.formatPlayObj({...req.body, connectionId: serverIdentifier});
const warnings = [];
if(req.body.ServerName === undefined || req.body.ServerName === '') {
if(req.body.ServerId !== undefined && req.body.ServerId !== '') {
warnings.push(`Webhook payload did not contain ServerName, will use ServerId (${req.body.ServerId}) instead`);
serverIdentifier = req.body.ServerId;
} else {
warnings.push(`Webhook payload did not contain ServerName OR ServerId will use connection ID (${serverIdentifier}) instead`);
}
} else {
serverIdentifier = playObj.meta.server;
}
if(this.seenServers[playObj.meta.server] === undefined) {
this.seenServers[playObj.meta.server] = playObj.meta.sourceVersion;
return [true, `Received valid data from server ${playObj.meta.server} (Version ${playObj.meta.sourceVersion}) for the first time.`];
if(this.seenServers[serverIdentifier] === undefined) {
let version = req.body.ServerVersion;
if(version === undefined || version === '') {
warnings.push('Webhook payload did not contain ServerVersion');
version = 'Unknown';
}
if(warnings.length > 0) {
this.logger.warn(`${remoteHostStr(req)} There is information missing from the Jellyfin webhook payload. Make sure 'Send all Properties' is checked in webhook: ${warnings.join(' | ')}`);
}
this.seenServers[serverIdentifier] = version;
return [true, `Received valid data from server ${serverIdentifier} (Version ${version}) for the first time.`];
}
}
return [true, undefined];
+1 -1
View File
@@ -18,7 +18,7 @@ export class PlexNotifier extends IngressNotifier {
}
const playObj = PlexSource.formatPlayObj(payload, true);
const playObj = PlexSource.formatPlayObj(payload, {newFromSource: true});
if(playObj.meta.server === undefined) {
return [false, `Payload from Plex did not contain server info! Check Plex logs for any errors.`];
@@ -12,7 +12,7 @@ export class TautulliNotifier extends IngressNotifier {
seenServers: string[] = [];
notifyBySource(req: Request, isRaw: boolean): [boolean, (string | undefined)] {
if(!isRaw) {
const playObj = TautulliSource.formatPlayObj(req, true);
const playObj = TautulliSource.formatPlayObj(req, {newFromSource: true});
if(!this.seenServers.includes(playObj.meta.server)) {
this.seenServers.push(playObj.meta.server);
+242 -72
View File
@@ -1,17 +1,21 @@
import {promises, constants} from "fs";
import {promises, constants, accessSync} from "fs";
import dayjs, {Dayjs} from 'dayjs';
import utc from 'dayjs/plugin/utc.js';
import winston, {Logger} from "winston";
import stringify from 'safe-stable-stringify';
import {Logger} from "winston";
import JSON5 from 'json5';
import { TimeoutError, WebapiError } from "spotify-web-api-node/src/response-error.js";
import { Response } from 'superagent';
import Ajv, {Schema} from 'ajv';
import {PlayObject, RemoteIdentityParts, TrackStringOptions} from "./common/infrastructure/Atomic.js";
import {
lowGranularitySources,
PlayObject, ProgressAwarePlayObject,
RemoteIdentityParts,
TrackStringOptions
} from "./common/infrastructure/Atomic.js";
import {Request} from "express";
const {format} = winston;
const {combine, printf, timestamp, label, splat, errors} = format;
import pathUtil from "path";
import {ErrorWithCause} from "pony-cause";
import backoffStrategies from '@kenyip/backoff-strategies';
import bstrat from "@kenyip/backoff-strategies";
dayjs.extend(utc);
@@ -88,8 +92,8 @@ export const buildTrackString = (playObj: PlayObject, options: TrackStringOption
transformers: {
artists: artistsFunc = (a: string[]) => a.join(' / '),
track: trackFunc = defaultTransformer,
time: timeFunc = (t: Dayjs) => t.local().format(),
timeFromNow = (t: Dayjs) => t.local().fromNow(),
time: timeFunc = (t: Dayjs | undefined) => t === undefined ? 'N/A' : t.local().format(),
timeFromNow = (t: Dayjs | undefined) => t === undefined ? undefined : t.local().fromNow(),
} = {}
} = options;
const {
@@ -100,13 +104,13 @@ export const buildTrackString = (playObj: PlayObject, options: TrackStringOption
playDate
} = {},
meta: {
sourceId
trackId
} = {},
} = playObj;
const strParts = [];
if(include.includes('sourceId') && sourceId !== undefined) {
strParts.push(`(${sourceId})`);
if(include.includes('trackId') && trackId !== undefined) {
strParts.push(`(${trackId})`);
}
if(include.includes('artist')) {
strParts.push(`${artistsFunc(artists)}`)
@@ -122,67 +126,61 @@ export const buildTrackString = (playObj: PlayObject, options: TrackStringOption
strParts.push(`@ ${timeFunc(playDate)}`);
}
if (include.includes('timeFromNow')) {
strParts.push(`(${timeFromNow(playDate)})`)
const tfn = timeFromNow(playDate);
if(tfn !== undefined) {
strParts.push(`(${tfn})`)
}
}
return strParts.join(' ');
}
// sorts playObj formatted objects by playDate in ascending (oldest first) order
export const sortByPlayDate = (a: any, b: any) => a.data.playDate.isAfter(b.data.playDate) ? 1 : -1;
const s = splat();
const SPLAT = Symbol.for('splat')
const errorsFormat = errors({stack: true});
const CWD = process.cwd();
let longestLabel = 3;
export const defaultFormat = printf(({level, message, label = 'App', timestamp, [SPLAT]: splatObj, stack, ...rest}) => {
let stringifyValue = splatObj !== undefined ? stringify(splatObj) : '';
if (label.length > longestLabel) {
longestLabel = label.length;
export const sortByOldestPlayDate = (a: PlayObject, b: PlayObject) => {
const {
data: {
playDate: aPlayDate
} = {}
} = a;
const {
data: {
playDate: bPlayDate
} = {}
} = b;
if(aPlayDate === undefined && bPlayDate === undefined) {
return 0;
}
let msg = message;
let stackMsg = '';
if (stack !== undefined) {
const stackArr = stack.split('\n');
msg = stackArr[0];
const cleanedStack = stackArr
.slice(1) // don't need actual error message since we are showing it as msg
.map((x: any) => x.replace(CWD, 'CWD')) // replace file location up to cwd for user privacy
.join('\n'); // rejoin with newline to preserve formatting
stackMsg = `\n${cleanedStack}`;
if(aPlayDate === undefined) {
return 1;
}
return `${timestamp} ${level.padEnd(7)}: [${label.padEnd(longestLabel)}] ${msg}${stringifyValue !== '' ? ` ${stringifyValue}` : ''}${stackMsg}`;
});
export const labelledFormat = (labelName = 'App') => {
const l = label({label: labelName, message: false});
return combine(
timestamp(
{
format: () => dayjs().local().format(),
}
),
l,
s,
errorsFormat,
defaultFormat,
);
}
export const createLabelledLogger = (name = 'default', label = 'App'): Logger => {
if (winston.loggers.has(name)) {
return winston.loggers.get(name);
if(bPlayDate === undefined) {
return -1;
}
const def = winston.loggers.get('default');
winston.loggers.add(name, {
transports: def.transports,
level: def.level,
format: labelledFormat(label)
});
return winston.loggers.get(name);
}
return aPlayDate.isAfter(bPlayDate) ? 1 : -1
};
export const sortByNewestPlayDate = (a: PlayObject, b: PlayObject) => {
const {
data: {
playDate: aPlayDate
} = {}
} = a;
const {
data: {
playDate: bPlayDate
} = {}
} = b;
if(aPlayDate === undefined && bPlayDate === undefined) {
return 0;
}
if(aPlayDate === undefined) {
return 1;
}
if(bPlayDate === undefined) {
return -1;
}
return aPlayDate.isBefore(bPlayDate) ? 1 : -1
};
export const setIntersection = (setA: any, setB: any) => {
let _intersection = new Set()
@@ -220,7 +218,7 @@ export const playObjDataMatch = (a: PlayObject, b: PlayObject) => {
} = {},
meta: {
source: aSource,
sourceId: aSourceId,
trackId: atrackId,
} = {},
} = a;
@@ -232,13 +230,13 @@ export const playObjDataMatch = (a: PlayObject, b: PlayObject) => {
} = {},
meta: {
source: bSource,
sourceId: bSourceId,
trackId: btrackId,
} = {},
} = b;
// if sources are the same and both plays have source ids then we can just compare by id
if(aSource === bSource && aSourceId !== undefined && bSourceId !== undefined) {
if(aSourceId !== bSourceId) {
if(aSource === bSource && atrackId !== undefined && btrackId !== undefined) {
if(atrackId !== btrackId) {
return false;
}
}
@@ -267,8 +265,8 @@ export const parseRetryAfterSecsFromObj = (err: any) => {
if (err instanceof TimeoutError) {
return undefined;
}
// @ts-ignore
if (err instanceof WebapiError || err instanceof Response) {
if (err instanceof WebapiError || 'headers' in err) {
const {headers = {}} = err;
raVal = headers['retry-after']
}
@@ -464,3 +462,175 @@ export const remoteHostStr = (req: Request): string => {
return `${host}${proxy !== undefined ? ` (${proxy})` : ''}${agent !== undefined ? ` (UA: ${agent})` : ''}`;
}
export const closePlayDate = (existingPlay: PlayObject, newPlay: PlayObject, options: { diffThreshold?: number, fuzzyDuration?: boolean} = {}): boolean => {
const {
meta:{
source,
},
data: {
playDate: existingPlayDate,
duration: existingDuration,
}
} = existingPlay;
const {
data: {
playDate: newPlayDate,
duration: newDuration,
}
} = newPlay;
const {
diffThreshold = lowGranularitySources.some(x => x.toLocaleLowerCase() === source) ? 60 : 10,
fuzzyDuration = false,
} = options;
// cant compare!
if(existingPlayDate === undefined || newPlayDate === undefined) {
return false;
}
const referenceDuration = newDuration ?? existingDuration;
let playDiffThreshold = diffThreshold;
let closeTime = false;
// check if existing play time is same as new play date
let scrobblePlayDiff = Math.abs(existingPlayDate.unix() - newPlayDate.unix());
if (scrobblePlayDiff <= playDiffThreshold) {
closeTime = true;
}
// if the source has a duration its possible one play was scrobbled at the beginning of the track and the other at the end
// so check if the duration matches the diff between the two play dates
if (closeTime === false && referenceDuration !== undefined && fuzzyDuration) {
if(Math.abs(scrobblePlayDiff - newDuration) < 10) { // use finer comparison for this
closeTime = true;
}
}
return closeTime;
}
export const combinePartsToString = (parts: any[], glue: string = '-'): string | undefined => {
const cleanParts: string[] = [];
for (const part of parts) {
if (part === null || part === undefined) {
continue;
}
if (Array.isArray(part)) {
const nestedParts = combinePartsToString(part, glue);
if (nestedParts !== undefined) {
cleanParts.push(nestedParts);
}
} else if (typeof part === 'object') {
// hope this works
cleanParts.push(JSON.stringify(part));
} else if(typeof part === 'string') {
if(part.trim() !== '') {
cleanParts.push(part);
}
} else {
cleanParts.push(part.toString());
}
}
if (cleanParts.length > 0) {
return cleanParts.join(glue);
}
return undefined;
}
/**
* Remove duplicates based on trackId, deviceId, and play date
* */
export const removeDuplicates = (plays: PlayObject[]): PlayObject[] => {
return plays.reduce((acc: PlayObject[], currPlay: PlayObject) => {
if(currPlay.meta.trackId !== undefined && currPlay.meta.deviceId !== undefined && currPlay.data.playDate !== undefined) {
if(acc.some((x: PlayObject) => x.meta.trackId === currPlay.meta.trackId && x.meta.deviceId === currPlay.meta.deviceId && x.data.playDate.isSame(currPlay.data.playDate, 'minute'))) {
// don't add current play to list if we find an existing that matches track, device, and play date
return acc;
}
}
return acc.concat(currPlay);
}, []);
}
export const toProgressAwarePlayObject = (play: PlayObject): ProgressAwarePlayObject => {
return {...play, meta: {...play.meta, initialTrackProgressPosition: play.meta.trackProgressPosition}};
}
export const getProgress = (initial: ProgressAwarePlayObject, curr: PlayObject): number | undefined => {
if(initial.meta.initialTrackProgressPosition !== undefined && curr.meta.trackProgressPosition !== undefined) {
return Math.abs(curr.meta.trackProgressPosition - initial.meta.initialTrackProgressPosition);
}
return undefined;
}
export function parseBool(value: any, prev: any = false): boolean {
let usedVal = value;
if (value === undefined || value === '') {
usedVal = prev;
}
if(usedVal === undefined || usedVal === '') {
return false;
}
if (typeof usedVal === 'string') {
return usedVal === 'true';
} else if (typeof usedVal === 'boolean') {
return usedVal;
}
throw new Error('Not a boolean value.');
}
export const genGroupId = (play: PlayObject) => `${play.meta.deviceId ?? 'NoDevice'}-${play.meta.user ?? 'SingleUser'}`;
export const fileOrDirectoryIsWriteable = (location: string) => {
const pathInfo = pathUtil.parse(location);
const isDir = pathInfo.ext === '';
try {
accessSync(location, constants.R_OK | constants.W_OK);
return true;
} catch (err: any) {
const {code} = err;
if (code === 'ENOENT') {
// file doesn't exist, see if we can write to directory in which case we are good
try {
accessSync(pathInfo.dir, constants.R_OK | constants.W_OK)
// we can write to dir
return true;
} catch (accessError: any) {
if(accessError.code === 'EACCES') {
// also can't access directory :(
throw new Error(`No ${isDir ? 'directory' : 'file'} exists at ${location} and application does not have permission to write to the parent directory`);
} else {
throw new ErrorWithCause(`No ${isDir ? 'directory' : 'file'} exists at ${location} and application is unable to access the parent directory due to a system error`, {cause: accessError});
}
}
} else if(code === 'EACCES') {
throw new Error(`${isDir ? 'Directory' : 'File'} exists at ${location} but application does not have permission to write to it.`);
} else {
throw new ErrorWithCause(`${isDir ? 'Directory' : 'File'} exists at ${location} but application is unable to access it due to a system error`, {cause: err});
}
}
}
export const mergeArr = (objValue: [], srcValue: []): (any[] | undefined) => {
if (Array.isArray(objValue)) {
return objValue.concat(srcValue);
}
}
export const pollingBackoff = (attempt: number, scaleFactor: number = 1): number => {
const backoffStrat = backoffStrategies({
delay: 1000,
strategy: "exponential",
jitter: true,
minimumDelay: 1000,
scaleFactor
});
// first attempt delay is never enough so always add + 1
return Math.round(backoffStrat(attempt + 1) / 1000);
}
-3
View File
@@ -38,9 +38,6 @@
<h3>Recently Played (<%= name %>)</h3>
</div>
<div class="p-6 md:px-10 md:py-6">
<% if (sourceType === 'spotify') { %>
This list was returned from the Spotify Api <b>now.</b> Sometimes Spotify "backlogs" user activity so your recent tracks may not show up for minutes (or hours). ¯\_(ツ)_/¯
<% } %>
<ul>
<% plays.forEach(function (play){ %>
<li><%-play%></li>
+20 -7
View File
@@ -18,6 +18,21 @@
.dark .loading {
fill: white;
}
.blue {
color: rgb(97, 175, 239)
}
.red {
color: rgb(255 123 133);
}
.green {
color: rgb(170 255 109);
}
.purple {
color: rgb(224 124 253);
}
.yellow {
color: rgb(253 198 94);
}
</style>
<title>Multi Scrobbler</title>
<meta charset="utf-8">
@@ -154,13 +169,11 @@
e.preventDefault()
fetch(e.target.href).then(req => {
if (req.ok) {
if (e.target.dataset.log === 'info') {
document.querySelector('[data-log="debug"]').classList.remove('font-bold', 'no-underline', 'pointer-events-none')
document.querySelector('[data-log="info"]').classList.add('font-bold', 'no-underline', 'pointer-events-none')
} else {
document.querySelector('[data-log="info"]').classList.remove('font-bold', 'no-underline', 'pointer-events-none')
document.querySelector('[data-log="debug"]').classList.add('font-bold', 'no-underline', 'pointer-events-none')
}
document.querySelectorAll('.log-level').forEach(elm => {
elm.classList.remove('font-bold', 'no-underline', 'pointer-events-none');
})
//document.querySelector('.log-level').classList.remove('font-bold', 'no-underline', 'pointer-events-none');
document.querySelector(`[data-log="${e.target.dataset.log}"]`).classList.add('font-bold', 'no-underline', 'pointer-events-none');
}
})
})
+1 -1
View File
@@ -4,7 +4,7 @@
"resolveJsonModule": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"lib": ["es2020"],
"lib": ["es2020", "dom"],
"module": "commonjs",
"target": "es2020",
"esModuleInterop": true,