Fix typo on users specification for jellyfin source setup

* Break up into different functions and add properties to concrete classes signalling its capabilities
* More detailed logging based on which fails/succeeds
* Simplify client/source init/auth step (no more need for case switch specifics other than creating object)

.dockerignore

@@ -5,5 +5,9 @@ Dockerfile
 .dockerignore
 .gitignore
 .git
-spotifyCreds.json
+config/currentCreds.json
 *.log
+config/maloja.json
+config/spotify.json
+config/config.json
+/docs

Dockerfile

@@ -1,5 +1,9 @@
 FROM node:fermium-alpine3.10
 
+ENV TZ=Etc/GMT
+
+RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+
 RUN mkdir -p /home/node/app/node_modules && chown -R node:node /home/node
 
 WORKDIR /home/node/app
@@ -8,7 +12,7 @@ COPY package*.json ./
 
 USER node
 
-RUN npm install
+RUN npm install --production
 
 COPY --chown=node:node . .
 
@@ -19,6 +23,11 @@ RUN mkdir -p $config_dir
 VOLUME $config_dir
 ENV CONFIG_DIR=$config_dir
 
+ARG log_dir=/home/node/logs
+RUN mkdir -p $log_dir
+VOLUME $log_dir
+ENV LOG_DIR=$log_dir
+
 ARG webPort=9078
 ENV PORT=$webPort
 EXPOSE $PORT

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 FoxxMD
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,83 +1,95 @@
-# spotify-scrobbler
+# multi-scrobbler
 
-A single-user, javascript app to scrobble your recent plays to [Maloja](https://github.com/krateng/maloja) (and other clients, eventually)
+[![Latest Release](https://img.shields.io/github/v/release/foxxmd/multi-scrobbler)](https://github.com/FoxxMD/multi-scrobbler/releases)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Docker Pulls](https://img.shields.io/docker/pulls/foxxmd/multi-scrobbler)](https://hub.docker.com/r/foxxmd/multi-scrobbler)
+
+A javascript app to scrobble plays from multiple sources to [Maloja](https://github.com/krateng/maloja), [Last.fm](https://www.last.fm), and other clients (eventually!)
+
+* Supports scrobbling for many sources
+  * [Spotify](/docs/configuration.md#spotify)
+  * [Plex](/docs/configuration.md#plex) or [Tautulli](/docs/configuration.md#tautulli)
+  * [Subsonic-compatible APIs](/docs/configuration.md#subsonic) (like [Airsonic](https://airsonic.github.io/))
+  * [Jellyfin](/docs/configuration.md#jellyfin)
+  * [Last.fm](/docs/configuration.md#lastfm-source)
+* Supports scrobbling to many clients
+  * [Maloja](/docs/configuration.md#maloja)
+  * [Last.fm](/docs/configuration.md#lastfm)
+* Supports configuring for single or multiple users (scrobbling for your friends and family!)
+* Web server interface for stats, basic control, and detailed logs
+* Smart handling of credentials (persistent, authorization through app)
+* Easy configuration through ENVs or JSON
+* Built for Docker and unattended use!
+
+**Why should I use this over a browser extension and/or mobile app scrobbler?**
+
+* **Platform independent** -- Because multi-scrobbler communicates directly with service APIs it will scrobble everything you play regardless of where you play it. No more need for apps on every platform you use!
+* **Open-source** -- Get peace of mind knowing exactly how your personal data is being handled.
+* **Consolidate play sources** -- Scrobble from many sources to one client with ease and without duplicating tracks.
+* **Manage scrobbling for others** -- Scrobble for your friends and family without any setup on their part. Easily silo sources to specific clients to keep plays separate.
+
+**But I already scrobble my music to Last.fm, is multi-scrobbler for me?**
+
+Yes! You can use [Last.fm as a Source](/docs/configuration.md#lastfm-source) to mirror scrobbles from your Last.fm profile to Maloja. That way you can keep your current scrobble setup as-is but still get the benefit of capturing your data to a self-hosted location.
+
+<img src="/assets/status-ui.jpg" width="800">
 
 ## Installation
 
+
+### Locally
+
 Clone this repository somewhere and then install from the working directory
 
 ```bash
+git clone https://github.com/FoxxMD/multi-scrobbler.git .
+cd multi-scrobbler
 npm install
 ```
 
-## Setup/Configuration
+### [Docker](https://hub.docker.com/r/foxxmd/multi-scrobbler)
 
-All configuration is done through json files or environment variables. Reference the [examples in the config folder](https://github.com/FoxxMD/spotify-scrobbler/tree/master/config) more detailed explanations and structure.
+```
+foxxmd/multi-scrobbler:latest
+```
 
-**A property from a json config will override the environmental variable.**
+## Setup
 
-### General
+Some setup is required! See the [configuration](docs/configuration.md) docs for a full reference.
 
-[JSON config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/config.json.example)
+### TLDR, Minimal Example
 
-Environment Variables
-* CONFIG_DIR - Default `./config` - Sets configuration directory to look for all other configuration files (if they are not specified)
-* CONFIG_PATH - Default `CONFIG_DIR/config.json`
-* LOG_PATH - Default `true` - If `false` no logs will be written. If `string` will be the directory logs are written to
-* PORT - Default 9078 - Port to run web server on (for authentication callbacks)
+You want to use multi-scrobbler to scrobble your plays from Spotify to Maloja:
 
+#### Local
+```bash
+SPOTIFY_CLIENT_ID=yourId SPOTIFY_CLIENT_SECRET=yourSecret MALOJA_URL=http://domain.tld MALOJA_API_KEY=1234 node index.js
+```
 
-### Spotify
+#### Docker
 
-[Spotify config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/spotify.json.example)
+```bash
+docker run -e "SPOTIFY_CLIENT_ID=yourId" -e "SPOTIFY_CLIENT_SECRET=yourSecret" -e "MALOJA_URL=http://domain.tld" -e "MALOJA_API_KEY=1234" -v /path/on/host/config:/home/node/app/config foxxmd/multi-scrobbler
+```
 
-Environment Variables
-* SPOTIFY_CONFIG_PATH - Optional, defaults to `CONFIG_DIR/spotify.json`
-* SPOTIFY_CLIENT_ID - **Required**
-* SPOTIFY_CLIENT_SECRET - **Required**
-* SPOTIFY_ACCESS_TOKEN - Optional if client/secret provided
-* SPOTIFY_REFRESH_TOKEN - Optional if client/secret provided
-* SPOTIFY_REDIRECT_URI - Optional, default is `http://localhost:{port}/callback`
+**But I want to use json for configuration?**
 
-The app will automatically obtain new access/refresh token if needed and possible. These will override values from configuration.
+Then use [config.json.example](/config/config.json.example) and drop it in your `CONFIG_DIR` directory
 
-### Maloja
+**Is there an example configuration using everything?**
 
-[Maloja config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/maloja.json.example)
-
-Environment Variables
-* MALOJA_CONFIG_PATH - Optional, defaults to `CONFIG_DIR/maloja.json`
-* MALOJA_URL - **Required** - Base Url of your Maloja installation
-* MALOJA_API_KEY - **Required** - Api Key for scrobbling
+Yes, check out the [kitchen sink example](/docs/kitchensink.md)
 
 ## Usage
 
-Output is provided to stdout/stderr as well as file if specified in configuration.
-
-On first startup you may need to authroize Spotify by visiting a callback URL. The default url to open is:
+A status page with statistics, recent logs, and some runtime configuration options can be found at
 
 ```
-https://localhost:9078/authSpotify
+https://localhost:9078
 ```
+Output is also provided to stdout/stderr as well as file if specified in configuration.
 
-### Running Directly
-
-```
-node index.js
-```
-
-### Docker
-
-[Docker repository](https://hub.docker.com/repository/docker/foxxmd/spotify-scrobbler)
-```
-foxxmd/spotify-scrobbler:latest
-```
-
-Minimal configuration requires you to bind a host directory for the configuration directory in the container:
-
-```
-docker run ... -v /path/on/host/config:/home/node/config ...
-```
+On first startup you may need to authorize Spotify by visiting the callback URL (which can also be accessed from the status page). Visit the status page above to find the applicable link to trigger this.
 
 ## License
 

apis/AbstractApiClient.js

@@ -0,0 +1,28 @@
+import {capitalize, createLabelledLogger} from "../utils.js";
+
+export default class AbstractApiClient {
+    name;
+    type;
+    initialized = false;
+
+    config;
+    options;
+    logger;
+
+    client;
+    workingCredsPath;
+    redirectUri;
+
+    constructor(type, name, config = {}, options = {}) {
+        this.type = type;
+        this.name = name;
+        const identifier = `API - ${capitalize(this.type)} - ${name}`;
+        this.logger = createLabelledLogger(identifier, identifier);
+        this.config = config;
+        this.options = options;
+    }
+
+    static formatPlayObj = obj => {
+        throw new Error('should be overridden');
+    }
+}

apis/LastfmApiClient.js

@@ -0,0 +1,164 @@
+import LastFm from "lastfm-node-client";
+import AbstractApiClient from "./AbstractApiClient.js";
+import dayjs from "dayjs";
+import {readJson, sleep, writeFile} from "../utils.js";
+
+const badErrors = [
+    'api key suspended',
+    'invalid session key',
+    'invalid api key',
+    'authentication failed'
+];
+
+const retryErrors = [
+    'operation failed',
+    'service offline',
+    'temporarily unavailable',
+    'rate limit'
+]
+
+export default class LastfmApiClient extends AbstractApiClient {
+
+    user;
+
+    constructor(name, config = {}, options = {}) {
+        super('lastfm', name, config, options);
+        const {redirectUri, apiKey, secret, session, configDir} = config;
+        this.redirectUri = `${redirectUri}?state=${name}`;
+        if (apiKey === undefined) {
+            this.logger.warn("'apiKey' not found in config!");
+        }
+        this.workingCredsPath = `${configDir}/currentCreds-lastfm-${name}.json`;
+        this.client = new LastFm(apiKey, secret, session);
+    }
+
+    static formatPlayObj = obj => {
+        const {
+            artist: {
+                // last.fm doesn't seem consistent with which of these properties it returns...
+                '#text': artists,
+                name: artistName,
+            },
+            name: title,
+            album: {
+                '#text': album,
+            },
+            duration,
+            date: {
+                uts: time,
+            } = {},
+            '@attr': {
+                nowplaying = 'false',
+            } = {},
+            url,
+            mbid,
+        } = obj;
+        // arbitrary decision yikes
+        let artistStrings = artists !== undefined ? artists.split(',') : [artistName];
+        return {
+            data: {
+                artists: [...new Set(artistStrings)],
+                track: title,
+                album,
+                duration,
+                playDate: time !== undefined ? dayjs.unix(time) : undefined,
+            },
+            meta: {
+                nowPlaying: nowplaying === 'true',
+                mbid,
+                source: 'Lastfm',
+                url: {
+                    web: url,
+                }
+            }
+        }
+    }
+
+    callApi = async (func, retries = 0) => {
+        const {
+            maxRequestRetries = 2,
+            retryMultiplier = 1.5
+        } = this.config;
+
+        try {
+            return await func(this.client);
+        } catch (e) {
+            const {
+                message,
+            } = e;
+            // for now check for exceptional errors by matching error code text
+            const retryError = retryErrors.find(x => message.toLocaleLowerCase().includes(x));
+            if(undefined !== retryError) {
+                if(retries < maxRequestRetries) {
+                    const delay = (retries + 1) * retryMultiplier;
+                    this.logger.warn(`API call was not good but recoverable (${retryError}), retrying in ${delay} seconds...`);
+                    await sleep(delay * 1000);
+                    return this.callApi(func, retries + 1);
+                } else {
+                    this.logger.warn('Could not recover!');
+                    throw e;
+                }
+            }
+
+            throw e;
+        }
+    }
+
+    getAuthUrl = () => {
+        const redir = `${this.config.redirectUri}?state=${this.name}`;
+        return `http://www.last.fm/api/auth/?api_key=${this.config.apiKey}&cb=${encodeURIComponent(redir)}`
+    }
+
+    authenticate = async (token) => {
+        const sessionRes = await this.client.authGetSession({token});
+        const {
+            session: {
+                key: sessionKey,
+                name, // username
+            } = {}
+        } = sessionRes;
+        this.client.sessionKey = sessionKey;
+
+        await writeFile(this.workingCredsPath, JSON.stringify({
+            sessionKey,
+        }));
+    }
+
+    initialize = async () => {
+
+        try {
+            const creds = await readJson(this.workingCredsPath, {throwOnNotFound: false});
+            const {sessionKey} = creds || {};
+            if (this.client.sessionKey === undefined && sessionKey !== undefined) {
+                this.client.sessionKey = sessionKey;
+            }
+            return true;
+        } catch (e) {
+            this.logger.warn('Current lastfm credentials file exists but could not be parsed', {path: this.workingCredsPath});
+            return false;
+        }
+    }
+
+    testAuth = async () => {
+        if (this.client.sessionKey === undefined) {
+            this.logger.info('No session key found. User interaction for authentication required.');
+            return false;
+        }
+        try {
+            const infoResp = await this.callApi(client => client.userGetInfo());
+            const {
+                user: {
+                    name,
+                } = {}
+            } = infoResp;
+            this.user = name;
+            this.initialized = true;
+            this.logger.info(`Client authorized for user ${name}`)
+            return true;
+        } catch (e) {
+            this.logger.error('Testing auth failed');
+            throw e;
+        }
+    }
+
+}

clients/AbstractScrobbleClient.js

@@ -0,0 +1,151 @@
+import dayjs from "dayjs";
+import {buildTrackString, capitalize, createLabelledLogger, playObjDataMatch} from "../utils.js";
+
+export default class AbstractScrobbleClient {
+
+    name;
+    type;
+    initialized = false;
+    requiresAuth = false;
+    requiresAuthInteraction = false;
+    authed = false;
+
+    recentScrobbles = [];
+    scrobbledPlayObjs = [];
+    newestScrobbleTime;
+    oldestScrobbleTime = dayjs();
+    tracksScrobbled = 0;
+
+    lastScrobbleCheck = dayjs();
+    refreshEnabled;
+    checkExistingScrobbles;
+    verboseOptions;
+
+    config;
+    logger;
+
+    constructor(type, name, config = {}) {
+        this.type = type;
+        this.name = name;
+        const identifier = `Client ${capitalize(this.type)} - ${name}`;
+        this.logger = createLabelledLogger(identifier, identifier);
+
+        const {
+            options: {
+                refreshEnabled = true,
+                checkExistingScrobbles = true,
+                verbose = {},
+            } = {},
+            ...rest
+        } = config;
+        this.config = rest;
+        this.refreshEnabled = refreshEnabled;
+        this.checkExistingScrobbles = checkExistingScrobbles;
+
+        const {
+            match: {
+                onNoMatch = false,
+                onMatch = false,
+                confidenceBreakdown = false,
+            } = {},
+            ...vRest
+        } = verbose
+        if (onMatch || onNoMatch) {
+            this.logger.warn('Setting verbose matching may produce noisy logs! Use with care.');
+        }
+        this.verboseOptions = {
+            ...vRest,
+            match: {
+                onNoMatch,
+                onMatch,
+                confidenceBreakdown
+            }
+        };
+    }
+
+    // default init function, should be overridden if init stage is required
+    initialize = async () => {
+        this.initialized = true;
+        return this.initialized;
+    }
+
+    // default init function, should be overridden if auth stage is required
+    testAuth = async () => {
+        return this.authed;
+    }
+
+    scrobblesLastCheckedAt = () => {
+        return this.lastScrobbleCheck;
+    }
+
+    formatPlayObj = obj => {
+        this.logger.warn('formatPlayObj should be defined by concrete class!');
+        return obj;
+    }
+
+    // time frame is valid as long as the play date for the source track is newer than the oldest play time from the scrobble client
+    // ...this is assuming the scrobble client is returning "most recent" scrobbles
+    timeFrameIsValid = (playObj, log = false) => {
+        const {
+            data: {
+                playDate,
+            } = {},
+        } = playObj;
+        const validTime = playDate.isAfter(this.oldestScrobbleTime);
+        if (log && !validTime) {
+            this.logger.debug(`${buildTrackString(playObj)} was in an invalid time frame (played before the oldest scrobble found)`);
+        }
+        return validTime;
+    }
+
+    addScrobbledTrack = (playObj, scrobbleResp) => {
+        this.scrobbledPlayObjs.push({play: playObj, scrobble: this.formatPlayObj(scrobbleResp)});
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+            } = {},
+        } = playObj;
+
+        return track;
+    };
+
+    findExistingSubmittedPlayObj = (playObj) => {
+        const {
+            data: {
+                playDate
+            } = {},
+            meta: {
+                source,
+            } = {}
+        } = playObj;
+
+        const dtInvariantMatches = this.scrobbledPlayObjs.filter(x => playObjDataMatch(playObj, x.play));
+
+        if (dtInvariantMatches.length === 0) {
+            return [undefined, undefined];
+        }
+
+        const matchPlayDate = dtInvariantMatches.find((x) => {
+            const {
+                play: {
+                    data: {
+                        playDate: sPlayDate
+                    } = {},
+                    meta: {
+                        source: playSource
+                    } = {},
+                } = {},
+            } = x;
+            // need to account for inaccurate DT from subsonic
+            if(source === 'Subsonic' && playSource === 'Subsonic') {
+                return playDate.isSame(sPlayDate) || playDate.diff(sPlayDate, 'minute') <= 1;
+            }
+            return playDate.isSame(sPlayDate);
+        });
+
+        return [matchPlayDate, dtInvariantMatches];
+    }
+}

clients/LastfmScrobbler.js

@@ -0,0 +1,311 @@
+import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
+import dayjs from 'dayjs';
+
+import {
+    buildTrackString,
+    playObjDataMatch,
+    setIntersection, sleep,
+    sortByPlayDate,
+    truncateStringToLength,
+} from "../utils.js";
+import LastfmApiClient from "../apis/LastfmApiClient.js";
+
+export default class LastfmScrobbler extends AbstractScrobbleClient {
+
+    api;
+    initialized = false;
+    requiresAuth = true;
+    requiresAuthInteraction = true;
+
+    constructor(name, config = {}, options = {}) {
+        super('lastfm', name, config, options);
+        this.api = new LastfmApiClient(name, config, options)
+    }
+
+    formatPlayObj = obj => LastfmApiClient.formatPlayObj(obj);
+
+    initialize = async () => {
+        this.initialized = await this.api.initialize();
+        return this.initialized;
+    }
+
+    testAuth = async () => {
+        try {
+            this.authed = await this.api.testAuth();
+        } catch (e) {
+            this.logger.error('Could not successfully communicate with Last.fm API');
+            this.logger.error(e);
+            this.authed = false;
+        }
+        return this.authed;
+    }
+
+    refreshScrobbles = async () => {
+        if (this.refreshEnabled) {
+            this.logger.debug('Refreshing recent scrobbles');
+            const resp = await this.api.callApi(client => client.userGetRecentTracks({user: this.api.user, limit: 20, extended: true}));
+            const {
+                recenttracks: {
+                    track: list = [],
+                }
+            } = resp;
+            this.recentScrobbles = list.reduce((acc, x) => {
+                try {
+                    const formatted = LastfmApiClient.formatPlayObj(x);
+                    const {
+                        data: {
+                            track,
+                            playDate,
+                        },
+                        meta: {
+                            mbid,
+                            nowPlaying,
+                        }
+                    } = formatted;
+                    if(nowPlaying === true) {
+                        // if the track is "now playing" it doesn't get a timestamp so we can't determine when it started playing
+                        // and don't want to accidentally count the same track at different timestamps by artificially assigning it 'now' as a timestamp
+                        // so we'll just ignore it in the context of recent tracks since really we only want "tracks that have already finished being played" anyway
+                        this.logger.debug("Ignoring 'now playing' track returned from Last.fm client", {track, mbid});
+                        return acc;
+                    } else if(playDate === undefined) {
+                        this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, {track, mbid});
+                        return acc;
+                    }
+                    return acc.concat(formatted);
+                } catch (e) {
+                    this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', {error: e.message});
+                    this.logger.debug('Full api response object:');
+                    this.logger.debug(x);
+                    return acc;
+                }
+            }, []).sort(sortByPlayDate);
+            if (this.recentScrobbles.length > 0) {
+                const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
+                const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
+                this.newestScrobbleTime = newestScrobbleTime;
+                this.oldestScrobbleTime = oldestScrobbleTime;
+
+                this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
+            }
+        }
+        this.lastScrobbleCheck = dayjs();
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+            } = {},
+        } = playObj;
+        return track.toLocaleLowerCase().trim();
+    }
+
+    alreadyScrobbled = (playObj, log = false) => {
+        return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
+    }
+
+    existingScrobble = (playObj, logMatch = false) => {
+        const tr = truncateStringToLength(27);
+        const scoreTrackOpts = {include: ['track', 'time'], transformers: {track: t => tr(t).padEnd(30)}};
+
+        // return early if we don't care about checking existing
+        if (false === this.checkExistingScrobbles) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
+            }
+            return undefined;
+        }
+
+        let existingScrobble;
+        let closestMatch = {score: 0, breakdowns: ['None']};
+
+        // then check if we have already recorded this
+        const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
+
+        // if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
+        if (existingExactSubmitted !== undefined) {
+            existingScrobble = existingExactSubmitted.scrobble;
+
+            closestMatch = {
+                score: 1,
+                breakdowns: ['Exact Match found in previously successfully scrobbled']
+            }
+        }
+        // if not though then we need to check recent scrobbles from scrobble api.
+        // this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
+
+        // if no recent scrobbles found then assume we haven't submitted it
+        // (either user doesnt want to check history or there is no history to check!)
+        if (this.recentScrobbles.length === 0) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
+            }
+            return undefined;
+        }
+
+        if (existingScrobble === undefined) {
+
+            // we have have found an existing submission but without an exact date
+            // in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
+            const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
+
+            const {
+                data: {
+                    artists: sourceArtists = [],
+                    playDate
+                } = {},
+                meta: {
+                    trackLength,
+                    source,
+                } = {},
+            } = playObj;
+
+            // clean source title so it matches title from the scrobble api response as closely as we can get it
+            let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
+
+            existingScrobble = this.recentScrobbles.find((x) => {
+
+                const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
+
+                const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
+
+                const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
+                let closeTime = false;
+                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
+                let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
+                let scrobblePlayStartDiff;
+                if (scrobblePlayDiff <= playDiffThreshold) {
+                    //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
+                    closeTime = true;
+                }
+                // also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
+                if (closeTime === false && trackLength !== undefined) {
+                    scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
+                    if (scrobblePlayStartDiff <= playDiffThreshold) {
+                        //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
+                        closeTime = true;
+                    }
+                }
+
+                let titleMatch;
+                const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
+                // because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
+                // and compare against length of scrobble title
+                const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter(x => x !== ''));
+                const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
+
+                titleMatch = commonTerms.size / sourceTitleTerms.size;
+
+                let artistMatch;
+                const lowerSourceArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+                const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
+                artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
+
+                const artistScore = .2 * artistMatch;
+                const titleScore = .3 * titleMatch;
+                const timeScore = .5 * (closeTime ? 1 : 0);
+                const referenceScore = .5 * (referenceMatch ? 1 : 0);
+                const score = artistScore + titleScore + timeScore;
+
+                let scoreBreakdowns = [
+                    `Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
+                    `Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
+                    `Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
+                    `Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
+                    `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+                ];
+
+                const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+
+                const scoreInfo = {
+                    score,
+                    scrobble: x,
+                    breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
+                }
+
+                if (closestMatch.score <= score && score > 0) {
+                    closestMatch = scoreInfo
+                }
+
+                return score >= .7;
+            });
+        }
+
+        if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
+            const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
+            this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
+        }
+        return existingScrobble;
+    }
+
+    scrobble = async (playObj) => {
+        const {
+            data: {
+                artists,
+                album,
+                track,
+                duration,
+                playDate
+            } = {},
+            data = {},
+            meta: {
+                source,
+                newFromSource = false,
+            } = {}
+        } = playObj;
+
+        const sType = newFromSource ? 'New' : 'Backlog';
+
+        try {
+            const response = await this.api.callApi(client => client.trackScrobble(
+                {
+                    artist: artists.join(', '),
+                    duration,
+                    track,
+                    album,
+                    timestamp: playDate.unix(),
+                }));
+            const {
+                scrobbles: {
+                    '@attr': {
+                        accepted = 0,
+                        ignored = 0,
+                        code,
+                    },
+                    scrobble: {
+                        track: {
+                           '#text': trackName,
+                        } = {},
+                        timestamp,
+                        ignoredMessage: {
+                            code: ignoreCode,
+                            '#text': ignoreMsg,
+                        } = {},
+                        ...rest
+                    } = {}
+                } = {},
+            } = response;
+            if(code === 5) {
+                this.initialized = false;
+                throw new Error('Service reported daily scrobble limit exceeded! 😬 Disabling client');
+            }
+            this.addScrobbledTrack(playObj, {...rest, date: { uts: timestamp}, name: trackName});
+            if (newFromSource) {
+                this.logger.info(`Scrobbled (New)     => (${source}) ${buildTrackString(playObj)}`);
+            } else {
+                this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
+            }
+            if(ignoreMsg !== '') {
+                this.logger.warn(`Service ignored this scrobble 😬 => (Code ${ignoreCode}) ${ignoreMsg}`)
+            }
+            // last fm has rate limits but i can't find a specific example of what that limit is. going to default to 1 scrobble/sec to be safe
+            await sleep(1000);
+        } catch (e) {
+            this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj)});
+            throw e;
+        }
+
+        return true;
+    }
+}

clients/MalojaScrobbler.js

@@ -1,74 +1,408 @@
-import ScrobbleClient from "./ScrobbleClient.js";
+import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
 import request from 'superagent';
-import format from 'date-fns/format/index.js';
+import dayjs from 'dayjs';
+import {
+    buildTrackString,
+    playObjDataMatch,
+    setIntersection,
+    sleep,
+    sortByPlayDate,
+    truncateStringToLength,
+    parseRetryAfterSecsFromObj
+} from "../utils.js";
 
-export default class MalojaScrobbler extends ScrobbleClient {
+const feat = ["ft.", "ft", "feat.", "feat", "featuring", "Ft.", "Ft", "Feat.", "Feat", "Featuring"];
 
-    name = 'Maloja';
+export default class MalojaScrobbler extends AbstractScrobbleClient {
 
-    refreshScrobbles = async () => {
-        const {url} = this.config;
-        const resp = await request.get(`${url}/apis/mlj_1/scrobbles?since=${format(new Date(), 'yyyy/MM/dd')}&to=${format(new Date(), 'yyyy/MM/dd')}`)
-        this.recentScrobbles = resp.body.list.slice(0, 10);
-        this.lastScrobbleCheck = new Date();
+    requiresAuth = true;
+
+    constructor(name, config = {}, options = {}) {
+        super('maloja', name, config, options);
+        const {url, apiKey} = config;
+        if (apiKey === undefined) {
+            this.logger.warn("'apiKey' not found in config! Client will most likely fail when trying to scrobble");
+        }
+        if (url === undefined) {
+            throw new Error("Missing 'url' for Maloja config");
+        }
     }
 
-    alreadyScrobbled = (title, playDate, duration) => {
-        const playUnix = playDate.getTime() / 1000;
-        const lowerTitle = title.toLocaleLowerCase();
-        return this.recentScrobbles.some((x) => {
-            const {time: scrobbleTime, title: scrobbleTitle} = x;
-            const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase();
-            if (lowerTitle.includes(lowerScrobbleTitle) || lowerScrobbleTitle.includes(lowerTitle)) {
-                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
-                let scrobblePlayDiff = Math.abs(playUnix - scrobbleTime);
-                if (scrobblePlayDiff < 10) {
-                    this.logger.debug(`Scrobble with same name found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`, {label: this.name});
-                    return true;
-                }
-                // also need to check that scrobble time isn't the BEGINNING of the track
-                let scrobblePlayStartDiff = Math.abs(playUnix - (scrobbleTime - duration));
-                if (scrobblePlayStartDiff < 10) {
-                    this.logger.debug(`Scrobble with same name found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`, {label: this.name});
-                    return true;
-                }
-                this.logger.debug(`Scrobble with same name found but the start/finish times vs scrobble time diffs were too large to consider dups (Start Diff ${scrobblePlayStartDiff}s) (End Diff ${scrobblePlayDiff}s)`, {label: this.name});
+    static formatPlayObj(obj) {
+        const {
+            artists,
+            title,
+            album,
+            duration,
+            time,
+        } = obj;
+        let artistStrings = artists.reduce((acc, curr) => {
+            let aString;
+            if (typeof curr === 'string') {
+                aString = curr;
+            } else if (typeof curr === 'object') {
+                aString = curr.name;
+            }
+            const aStrings = aString.split(',');
+            return [...acc, ...aStrings];
+        }, []);
+        return {
+            data: {
+                artists: [...new Set(artistStrings)],
+                track: title,
+                album,
+                duration,
+                playDate: dayjs.unix(time),
+            },
+            meta: {
+                source: 'Maloja',
+            }
+        }
+    }
+
+    formatPlayObj = obj => MalojaScrobbler.formatPlayObj(obj);
+
+    callApi = async (req, retries = 0) => {
+        const {
+            maxRequestRetries = 1,
+            retryMultiplier = 1.5
+        } = this.config;
+
+        try {
+            return await req;
+        } catch (e) {
+            if(retries < maxRequestRetries) {
+                const retryAfter = parseRetryAfterSecsFromObj(e) ?? (retryMultiplier * (retries + 1));
+                this.logger.warn(`Request failed but retries (${retries}) less than max (${maxRequestRetries}), retrying request after ${retryAfter} seconds...`);
+                await sleep(retryAfter * 1000);
+                return await this.callApi(req, retries + 1)
+            }
+            const {
+                message,
+                response: {
+                    status,
+                    body,
+                    text,
+                } = {},
+                response,
+            } = e;
+            let msg = response !== undefined ? `API Call failed: Server Response => ${message}` : `API Call failed: ${message}`;
+            const responseMeta = body ?? text;
+            this.logger.error(msg, {status, response: responseMeta});
+            throw e;
+        }
+    }
+
+    testConnection = async () => {
+
+        const {url} = this.config;
+        try {
+            const serverInfoResp = await this.callApi(request.get(`${url}/apis/mlj_1/serverinfo`));
+            const {
+                statusCode,
+                body: {
+                    version = [],
+                    versionstring = '',
+                } = {},
+            } = serverInfoResp;
+
+            if (statusCode >= 300) {
+                this.logger.info('Test connection failed');
                 return false;
             }
+
+            this.logger.info('Test connection succeeded!');
+
+            if (version.length === 0) {
+                this.logger.warn('Server did not respond with a version. Either the base URL is incorrect or this Maloja server is too old :(');
+            } else {
+                this.logger.info(`Maloja Server Version: ${versionstring}`);
+                if (version[0] < 2 || version[1] < 7) {
+                    this.logger.warn('Maloja Server Version is less than 2.7, please upgrade to ensure compatibility');
+                }
+            }
+            return true;
+        } catch (e) {
+            this.logger.error('Testing connection failed');
+            this.logger.error(e);
             return false;
-        })
+        }
+    }
+
+    initialize = async () => {
+        // just checking that we can get a connection
+        this.initialized = await this.testConnection();
+        return this.initialized;
+    }
+
+    testAuth = async (withKey = true) => {
+
+        const {url, apiKey} = this.config;
+        try {
+            const resp = await this.callApi(request
+                .get(`${url}/apis/mlj_1/test`)
+                .query({key: apiKey}));
+
+            const {
+                status,
+                body: {
+                    status: bodyStatus,
+                } = {},
+                body = {},
+                text = '',
+            } = resp;
+            if (bodyStatus.toLocaleLowerCase() === 'ok') {
+                this.logger.info('Auth test passed!');
+                this.authed = true;
+            } else {
+                this.authed = false;
+                this.logger.error('Testing connection failed => Server Response body was malformed -- should have returned "status: ok"...is the URL correct?', {
+                    status,
+                    body,
+                    text: text.slice(0, 50)
+                });
+            }
+        } catch (e) {
+            this.logger.error('Auth test failed');
+            this.logger.error(e);
+            this.authed = false;
+        }
+        return this.authed;
+    }
+
+    refreshScrobbles = async () => {
+        if (this.refreshEnabled) {
+            this.logger.debug('Refreshing recent scrobbles');
+            const {url} = this.config;
+            const resp = await this.callApi(request.get(`${url}/apis/mlj_1/scrobbles?max=20`));
+            const {
+                body: {
+                    list = [],
+                } = {},
+            } = resp;
+            this.recentScrobbles = list.map(x => MalojaScrobbler.formatPlayObj(x)).sort(sortByPlayDate);
+            if (this.recentScrobbles.length > 0) {
+                const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
+                const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
+                this.newestScrobbleTime = newestScrobbleTime;
+                this.oldestScrobbleTime = oldestScrobbleTime;
+
+                this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
+            }
+        }
+        this.lastScrobbleCheck = dayjs();
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+                artists: sourceArtists = [],
+            } = {},
+        } = playObj;
+        let lowerTitle = track.toLocaleLowerCase();
+        lowerTitle = feat.reduce((acc, curr) => acc.replace(curr, ''), lowerTitle);
+        // also remove [artist] from the track if found since that gets removed as well
+        const lowerArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+        lowerTitle = lowerArtists.reduce((acc, curr) => acc.replace(curr, ''), lowerTitle);
+
+        // remove any whitespace in parenthesis
+        lowerTitle = lowerTitle.replace("\\s+(?=[^()]*\\))", '')
+            // replace parenthesis
+            .replace('()', '')
+            .replace('( )', '')
+            .trim();
+
+        return lowerTitle;
+    }
+
+    alreadyScrobbled = (playObj, log = false) => {
+        return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
+    }
+
+    existingScrobble = (playObj, logMatch = false) => {
+        const tr = truncateStringToLength(27);
+        const scoreTrackOpts = {include: ['track', 'time'], transformers: {track: t => tr(t).padEnd(30)}};
+
+        // return early if we don't care about checking existing
+        if (false === this.checkExistingScrobbles) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
+            }
+            return undefined;
+        }
+
+        let existingScrobble;
+        let closestMatch = {score: 0, breakdowns: ['None']};
+
+        // then check if we have already recorded this
+        const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
+
+        // if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
+        if (existingExactSubmitted !== undefined) {
+            existingScrobble = existingExactSubmitted.scrobble;
+
+            closestMatch = {
+                score: 1,
+                breakdowns: ['Exact Match found in previously successfully scrobbled']
+            }
+        }
+        // if not though then we need to check recent scrobbles from scrobble api.
+        // this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
+
+        // if no recent scrobbles found then assume we haven't submitted it
+        // (either user doesnt want to check history or there is no history to check!)
+        if (this.recentScrobbles.length === 0) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
+            }
+            return undefined;
+        }
+
+        if (existingScrobble === undefined) {
+
+            // we have have found an existing submission but without an exact date
+            // in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
+            const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
+
+            const {
+                data: {
+                    artists: sourceArtists = [],
+                    playDate
+                } = {},
+                meta: {
+                    trackLength,
+                    source,
+                } = {},
+            } = playObj;
+
+            // clean source title so it matches title from the scrobble api response as closely as we can get it
+            let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
+
+            existingScrobble = this.recentScrobbles.find((x) => {
+
+                const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
+
+                const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
+
+                const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
+                let closeTime = false;
+                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
+                let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
+                let scrobblePlayStartDiff;
+                if (scrobblePlayDiff <= playDiffThreshold) {
+                    //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
+                    closeTime = true;
+                }
+                // also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
+                if (closeTime === false && trackLength !== undefined) {
+                    scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
+                    if (scrobblePlayStartDiff <= playDiffThreshold) {
+                        //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
+                        closeTime = true;
+                    }
+                }
+
+                let titleMatch;
+                const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
+                // because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
+                // and compare against length of scrobble title
+                const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter(x => x !== ''));
+                const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
+
+                titleMatch = commonTerms.size / sourceTitleTerms.size;
+
+                let artistMatch;
+                const lowerSourceArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+                const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
+                artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
+
+                const artistScore = .2 * artistMatch;
+                const titleScore = .3 * titleMatch;
+                const timeScore = .5 * (closeTime ? 1 : 0);
+                const referenceScore = .5 * (referenceMatch ? 1 : 0);
+                const score = artistScore + titleScore + timeScore;
+
+                let scoreBreakdowns = [
+                    `Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
+                    `Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
+                    `Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
+                    `Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
+                    `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+                ];
+
+                const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+
+                const scoreInfo = {
+                    score,
+                    scrobble: x,
+                    breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
+                }
+
+                if (closestMatch.score <= score && score > 0) {
+                    closestMatch = scoreInfo
+                }
+
+                return score >= .7;
+            });
+        }
+
+        if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
+            const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
+            this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
+        }
+        return existingScrobble;
     }
 
     scrobble = async (playObj) => {
         const {url, apiKey} = this.config;
 
         const {
-            track: {
-                artists = [],
-                name,
-                id,
-                external_urls: {
-                    spotify,
-                } = {}
+            data: {
+                artists,
+                album,
+                track,
+                duration,
+                playDate
             } = {},
-            played_at
+            meta: {
+                source,
+                newFromSource = false,
+            } = {}
         } = playObj;
 
-        let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(',');
-        const time = new Date(played_at);
+        const sType = newFromSource ? 'New' : 'Backlog';
+
         try {
-            await request.post(`${url}/apis/mlj_1/newscrobble`)
+            const response = await this.callApi(request.post(`${url}/apis/mlj_1/newscrobble`)
                 .type('json')
                 .send({
-                    artist: artistString,
-                    title: name,
+                    // maloja seems to detect this deliminator much better than commas
+                    // also less likely artist has a forward slash in their name than a comma
+                    artist: artists.join(' / '),
+                    seconds: duration,
+                    title: track,
+                    album,
                     key: apiKey,
-                    time: time.getTime() / 1000
-                });
-            this.logger.info('Scrobbled', {label: this.name});
+                    time: playDate.unix(),
+                }));
+            const {body: {
+                track: {
+                    time: mTime = playDate.unix(),
+                    duration: mDuration = duration,
+                    album: mAlbum = album,
+                    ...rest
+                }
+            } = {}} = response;
+            this.addScrobbledTrack(playObj, {...rest, album: mAlbum, time: mTime, duration: mDuration});
+            if (newFromSource) {
+                this.logger.info(`Scrobbled (New)     => (${source}) ${buildTrackString(playObj)}`);
+            } else {
+                this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
+            }
         } catch (e) {
-            this.logger.error('Error while scrobbling', {label: this.name});
-            this.logger.log(e);
+            this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj)});
+            throw e;
         }
 
         return true;

clients/ScrobbleClient.js

@@ -1,20 +0,0 @@
-export default class ScrobbleClient {
-
-    name;
-
-    recentScrobbles = [];
-
-    lastScrobbleCheck = new Date();
-
-    config;
-    logger;
-
-    constructor(logger, config = {}) {
-        this.logger = logger;
-        this.config = config;
-    }
-
-    scrobblesLastCheckedAt = () => {
-        return this.lastScrobbleCheck;
-    }
-}

clients/ScrobbleClients.js

@@ -0,0 +1,323 @@
+import dayjs from "dayjs";
+import {
+    createLabelledLogger,
+    isValidConfigStructure,
+    playObjDataMatch,
+    readJson,
+    returnDuplicateStrings
+} from "../utils.js";
+import MalojaScrobbler from "./MalojaScrobbler.js";
+import LastfmScrobbler from "./LastfmScrobbler.js";
+
+export default class ScrobbleClients {
+
+    clients = [];
+    logger;
+    configDir;
+
+    clientTypes = ['maloja','lastfm'];
+
+    constructor(configDir) {
+        this.configDir = configDir;
+        this.logger = createLabelledLogger('scrobblers', 'Scrobblers');
+    }
+
+    getByName = (name) => {
+        return this.clients.find(x => x.name === name);
+    }
+
+    getByType = (type) => {
+        return this.clients.filter(x => x.type === type);
+    }
+
+    buildClientsFromConfig = async () => {
+        let configs = [];
+
+        let configFile;
+        try {
+            configFile = await readJson(`${this.configDir}/config.json`, {throwOnNotFound: false});
+        } catch (e) {
+            // think this should stay as show-stopper since config could include important defaults (delay, retries) we don't want to ignore
+            throw new Error('config.json could not be parsed');
+        }
+        let clientDefaults = {};
+        if (configFile !== undefined) {
+            const {
+                clients: mainConfigClientConfigs = [],
+                clientDefaults: cd = {},
+            } = configFile;
+            clientDefaults = cd;
+            const validMainConfigs = mainConfigClientConfigs.reduce((acc, curr, i) => {
+                if(curr === null) {
+                    this.logger.error(`The client config entry at index ${i} in config.json is null but should be an object, will not parse`);
+                    return acc;
+                }
+                if(typeof curr !== 'object') {
+                    this.logger.error(`The client config entry at index ${i} in config.json should be an object, will not parse`);
+                    return acc;
+                }
+                return acc.concat(curr);
+            }, []);
+            for (const c of validMainConfigs) {
+                const {name = 'unnamed'} = c;
+                configs.push({...c,
+                    name,
+                    source: 'config.json',
+                    configureAs: 'client', //override user value
+                });
+            }
+        }
+
+        for (const clientType of this.clientTypes) {
+            let defaultConfigureAs = 'client';
+            switch (clientType) {
+                case 'maloja':
+                    // env builder for single user mode
+                    const url = process.env.MALOJA_URL;
+                    const apiKey = process.env.MALOJA_API_KEY;
+                    if (url !== undefined || apiKey !== undefined) {
+                        configs.push({
+                            type: 'maloja',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: {
+                                url,
+                                apiKey
+                            }
+                        })
+                    }
+                    break;
+                case 'lastfm':
+                    const lfm = {
+                        apiKey: process.env.LASTFM_API_KEY,
+                        secret: process.env.LASTFM_SECRET,
+                        redirectUri: process.env.LASTFM_REDIRECT_URI,
+                        session: process.env.LASTFM_SESSION,
+                    };
+                    if (!Object.values(lfm).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'lastfm',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: lfm
+                        })
+                    }
+                    break;
+                default:
+                    break;
+            }
+            let rawClientConfigs;
+            try {
+                rawClientConfigs = await readJson(`${this.configDir}/${clientType}.json`, {throwOnNotFound: false});
+            } catch (e) {
+                this.logger.error(`${clientType}.json config file could not be parsed`);
+                continue;
+            }
+            if (rawClientConfigs !== undefined) {
+                let clientConfigs = [];
+                if (Array.isArray(rawClientConfigs)) {
+                    clientConfigs = rawClientConfigs;
+                } else if(rawClientConfigs === null) {
+                    this.logger.error(`${clientType}.json contained no data`);
+                    continue;
+                } else if (typeof rawClientConfigs === 'object') {
+                    // backwards compatibility, assuming its single-user mode
+                    this.logger.warn(`DEPRECATED: Starting in 0.4 configurations in all [type].json files (${clientType}.json) must be in an array.`);
+                    if (rawClientConfigs.data === undefined) {
+                        clientConfigs = [{data: rawClientConfigs, mode: 'single', name: 'unnamed'}];
+                    } else {
+                        clientConfigs = [rawClientConfigs];
+                    }
+                } else {
+                    this.logger.error(`All top level data from ${clientType}.json must be an array of objects, will not parse configs from file`);
+                    continue;
+                }
+                for (const [i,m] of clientConfigs.entries()) {
+                    if(m === null) {
+                        this.logger.error(`The config entry at index ${i} from ${clientType}.json is null`);
+                        continue;
+                    }
+                    if (typeof m !== 'object') {
+                        this.logger.error(`The config entry at index ${i} from ${clientType}.json was not an object, skipping`, m);
+                        continue;
+                    }
+                    const {configureAs = defaultConfigureAs} = m;
+                    if(configureAs === 'client') {
+                        m.source = `${clientType}.json`;
+                        m.type = clientType;
+                        configs.push(m);
+                    }
+                }
+            }
+        }
+
+        // we have all possible client configurations so we'll check they are minimally valid
+        const validConfigs = configs.reduce((acc, c) => {
+            const isValid = isValidConfigStructure(c, {type: true, data: true});
+            if (isValid !== true) {
+                this.logger.error(`Client config from ${c.source} with name [${c.name || 'unnamed'}] of type [${c.type || 'unknown'}] will not be used because it has structural errors: ${isValid.join(' | ')}`);
+                return acc;
+            }
+            return acc.concat(c);
+        }, []);
+
+        // all client configs are minimally valid
+        // now check that names are unique
+        const nameGroupedConfigs = validConfigs.reduce((acc, curr) => {
+            const {name = 'unnamed'} = curr;
+            const {[name]: n = []} = acc;
+            return {...acc, [name]: [...n, curr]};
+        }, {});
+        let noConflictConfigs = [];
+        for (const [name, configs] of Object.entries(nameGroupedConfigs)) {
+            if (configs.length > 1) {
+                const sources = configs.map(c => `Config object from ${c.source} of type [${c.type}]`);
+                this.logger.error(`The following clients will not be built because of config naming conflicts (they have the same name of "${name}"): 
+${sources.join('\n')}`);
+                if (name === 'unnamed') {
+                    this.logger.info('HINT: "unnamed" configs occur when using ENVs, if a multi-user mode config does not have a "name" property, or if a config is built in single-user mode');
+                }
+            } else {
+                noConflictConfigs = [...noConflictConfigs, ...configs];
+            }
+        }
+
+        // finally! all configs are valid, structurally, and can now be passed to addClient
+        // just need to re-map unnnamed to default
+        const finalConfigs = noConflictConfigs.map(({name = 'unnamed', ...x}) => ({
+            ...x,
+            name
+        }));
+        for (const c of finalConfigs) {
+            try {
+                await this.addClient(c, clientDefaults);
+            } catch(e) {
+                this.logger.error(`Client ${c.name} was not added because it had unrecoverable errors`);
+                this.logger.error(e);
+            }
+        }
+    }
+
+    addClient = async (clientConfig, defaults = {}) => {
+        const isValidConfig = isValidConfigStructure(clientConfig, {name: true, data: true, type: true});
+        if (isValidConfig !== true) {
+            throw new Error(`Config object from ${clientConfig.source || 'unknown'} with name [${clientConfig.name || 'unnamed'}] of type [${clientConfig.type || 'unknown'}] has errors: ${isValidConfig.join(' | ')}`)
+        }
+        const {type, name, data: d = {}} = clientConfig;
+        // add defaults
+        const data = {...defaults, ...d};
+        let newClient;
+        this.logger.debug(`(${name}) Constructing ${type} client...`);
+        switch (type) {
+            case 'maloja':
+                newClient = new MalojaScrobbler(name, data);
+                break;
+            case 'lastfm':
+                newClient = new LastfmScrobbler(name, {...data, configDir: this.configDir});
+                break;
+            default:
+                break;
+        }
+
+        if(newClient === undefined) {
+            // really shouldn't get here!
+            throw new Error(`Client of type ${type} was not recognized??`);
+        }
+        if(newClient.initialized === false) {
+            this.logger.debug(`(${name}) Attempting ${type} initialization...`);
+            if (await newClient.initialize() === false) {
+                this.logger.error(`(${name}) ${type} client failed to initialize. Client needs to be successfully initialized before scrobbling.`);
+            } else {
+                this.logger.info(`(${name}) ${type} client initialized`);
+            }
+        }
+        if(newClient.requiresAuth && !newClient.authed) {
+            this.logger.debug(`(${name}) Checking ${type} client auth...`);
+            let success;
+            try {
+                success = await newClient.testAuth();
+            } catch (e) {
+                success = false;
+            }
+            if(!success) {
+                this.logger.warn(`(${name}) ${type} client auth failed.`);
+            } else {
+                this.logger.warn(`(${name}) ${type} client auth OK`);
+            }
+        }
+        this.clients.push(newClient);
+    }
+
+    /**
+     * @param {*} data
+     * @param {{scrobbleFrom, scrobbleTo, forceRefresh: boolean}|{scrobbleFrom, scrobbleTo}} options
+     * @returns {Array}
+     */
+    scrobble = async (data, options = {}) => {
+        const playObjs = Array.isArray(data) ? data : [data];
+        const {
+            forceRefresh = false,
+            checkTime = dayjs(),
+            scrobbleTo = [],
+            scrobbleFrom = 'source',
+        } = options;
+
+        const tracksScrobbled = [];
+
+        if (this.clients.length === 0) {
+            this.logger.warn('Cannot scrobble! No clients are configured.');
+        }
+
+        for (const client of this.clients) {
+            if (scrobbleTo.length > 0 && !scrobbleTo.includes(client.name)) {
+                this.logger.debug(`Client '${client.name}' was filtered out by '${scrobbleFrom}'`);
+                continue;
+            }
+            if(client.initialized === false) {
+                this.logger.warn(`Cannot scrobble to Client '${client.name}' because it is not yet initialized`);
+                continue;
+            }
+            if(client.requiresAuthInteraction === true && !client.authed) {
+                this.logger.warn(`Cannot scrobble to Client '${client.name}' because user interaction is required for authentication`);
+                continue;
+            }
+
+                if (forceRefresh || client.scrobblesLastCheckedAt().unix() < checkTime.unix()) {
+                    try {
+                        await client.refreshScrobbles();
+                    } catch(e) {
+                        this.logger.error(`Encountered error while refreshing scrobbles for ${client.name}`);
+                        this.logger.error(e);
+                    }
+                }
+                for (const playObj of playObjs) {
+                    try {
+                        const {
+                            meta: {
+                                newFromSource = false,
+                            } = {}
+                        } = playObj;
+                        if (client.timeFrameIsValid(playObj, newFromSource) && !client.alreadyScrobbled(playObj, newFromSource)) {
+                            await client.scrobble(playObj)
+                            client.tracksScrobbled++;
+                            // since this is what we return to the source only add to tracksScrobbled if not already in array
+                            // (source should only know that a track was scrobbled (binary) -- doesn't care if it was scrobbled more than once
+                            if(!tracksScrobbled.some(x => playObjDataMatch(x, playObj) && x.data.playDate === playObj.data.playDate)) {
+                                tracksScrobbled.push(playObj);
+                            }
+                        }
+                    } catch(e) {
+                        this.logger.error(`Encountered error while in scrobble loop for ${client.name}`);
+                        this.logger.error(e);
+                        // for now just stop scrobbling plays for this client and move on. the client should deal with logging the issue
+                        if(e.continueScrobbling !== true) {
+                            break;
+                        }
+                    }
+                }
+        }
+        return tracksScrobbled;
+    }
+}

config/config.json.example

@@ -1,23 +1,32 @@
 {
-  // string specifies a directory location to log to.
-  // boolean FALSE means do not log to file
-  // boolean TRUE means log to working directory
-  "logPath": true, // optional
-  "interval": 60, // optional, number of seconds to wait before checking spotify for new tracks
-  "port": 9078, // optional, port for server to start on
-  "spotify": 'object or string', // can specify full spotify config object here OR a string location of json file OR do not include to use config dir location
-  // array of client configurations
+  "sourceDefaults": {
+    "maxPollRetries": 0,          // optional, default # of automatic polling restarts on error. can be overridden by property in individual config
+    "maxRequestRetries": 1,       // optional, default # of http request retries a source can make before error is thrown. can be overridden by property in individual config
+    "retryMultiplier": 1.5,       // optional, default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying). can be overridden by property in individual config
+  },
+  "clientDefaults": {
+    "maxRequestRetries": 1,       // optional, default # of http request retries a client can make before error is thrown. can be overridden by property in individual config
+    "retryMultiplier": 1.5,       // optional, default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying). can be overridden by property in individual config
+  },
+  "sources": [
+    {
+      "type": "spotify",          // required, source type
+      "clients": ["myConfig"],    // optional, a list of Client config names this Source should scrobble to. Using an empty list or not including this property will make this Source scrobble to all Clients.
+      "name": "mySpotifySource",  // optional, friendly name for the log
+      "data": {                   // required, the data for your config
+        "clientId": "example",
+        //...
+      }
+    }
+  ],
   "clients": [
     {
-      "type": "maloja", // client name
-      "data": "/someLocation/aFile.json" // location to look for json configuration in
-    },
-    {
-      "type": "maloja", // client name
-      "data": {
-        // client configuration can also be an object
+      "type": "maloja",           // required, Client type
+      "name": "myConfig",         // required, a name to identifier your Client
+      "data": {                   // required, the data for your config
+        "url": "http://example.com",
+        //...
       }
     },
-    "maloja" // client can also be just the type if the configuration is located in the default configuration directory
   ]
 }

config/jellyfin.json.example

@@ -0,0 +1,10 @@
+[
+  {
+    "name": "default",                                        // optional, friendly name for logs
+    "clients": [],                                            // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "users": ["FoxxMD"],                                     // optional, list of users to scrobble tracks for
+      "servers": ["myServer","anotherServer"]                 // optional, list of servers to scrobble tracks from
+    }
+  }
+]

config/lastfm.json.example

@@ -0,0 +1,16 @@
+[
+  {
+    "name": "myLastFm",                                          // [As Client/Source] required if configured as "client", a name to identify your Client/Source
+    "configureAs": "client",                                     // optional and default to "client", set to "source" to use this configuration as a Source
+    "clients": [],                                               // [As Source] optional, list of scrobble Clients (by config name) that this Source should scrobble to. Using an empty list or not including this property will make this Source scrobble to all Clients.
+    "data": {
+      "apiKey": "string",                                        // required, Lastfm api key
+      "secret": "string",                                        // required, Lastfm shared secret
+      "session": "string",                                       // optional, session id returned from a complete auth flow.
+                                                                 //   if not specified will be generated during authentication
+      "redirectUri": "http://localhost:9078/lastfm/callback"     // optional, if not different than this default
+                                                                 //   callback for auth. Must have "lastfm/callback" in the url somewhere
+      // ALSO see config.json.example for default properties that can be overridden here (in clientDefaults)
+    }
+  }
+]

config/maloja.json.example

@@ -1,4 +1,10 @@
-{
-  "url": "https://domain.tld", // the base url of your maloja installation
-  "apiKey": "string" // your maloja api key
-}
+[
+  {
+    "name": "myMaloja",            // required, a name to identify your Client
+    "data": {
+      "url": "https://domain.tld", // required, the base url of your maloja installation
+      "apiKey": "string"           // required, your maloja api key
+      // ALSO see config.json.example for default properties that can be overridden here (in clientDefaults)
+    }
+  }
+]

config/plex.json.example

@@ -0,0 +1,11 @@
+[
+  {
+    "name": "default",                                        // optional, friendly name for logs
+    "clients": [],                                            // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "user": ["username@gmail.com","anotherUser@gmail.com"], // optional, list of users to scrobble tracks for
+      "libraries": ["music","my podcasts"],                   // optional, list of libraries to scrobble tracks from
+      "servers": ["myServer","anotherServer"]                 // optional, list of servers to scrobble tracks from
+    }
+  }
+]

config/spotify.json.example

@@ -1,6 +1,15 @@
-{
-  "clientId": "string", // spotify client id
-  "clientSecret": "string", // spotify client secret
-  "redirectUri": "http://localhost:9078/callback", // optional, spotify callback url. useful if this installation is behind a reverse proxy
-  "callbackPath": "callback" // optional, if redirectUri does not end with /callback then must specify path here
-}
+[
+  {
+    "name": "default",                                // optional, friendly name for logs
+    "clients": [],                                    // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "clientId": "string",                           // spotify client id -- required if not providing access token
+      "clientSecret": "string",                       // spotify client secret -- required if not providing access token
+      "accessToken": "string",                        // spotify access token -- required if not providing client id/secret
+      "refreshToken": "string",                       // spotify refresh token -- recommended to provide if not providing client id/secret
+      "redirectUri": "http://localhost:9078/callback",// spotify redirect URI -- required only if not the default shown here. URI must end in "callback"
+      "interval": 60,                                 // optional, how long to wait before calling spotify for new tracks
+      // ALSO see config.json.example for default properties that can be overridden here (in sourceDefaults)
+    }
+  }
+]

config/subsonic.json.example

@@ -0,0 +1,11 @@
+[
+  {
+    "name": "default",                        // optional, friendly name for logs
+    "data": {
+      "url": "http://localhost:4040/airsonic",// required, the url you would visit to listen to music on the web
+      "user": "yourUser",                     // required, username to login with
+      "password": "yourPassword",             // required, password to login with
+      // ALSO see config.json.example for default properties that can be overridden here (in sourceDefaults)
+    }
+  }
+]

config/tautulli.json.example

@@ -0,0 +1,11 @@
+[
+  {
+    "name": "default",                                        // optional, friendly name for logs
+    "clients": [],                                            // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "user": ["username@gmail.com","anotherUser@gmail.com"], // optional, list of users to scrobble tracks for
+      "libraries": ["music","my podcasts"],                   // optional, list of libraries to scrobble tracks from
+      "servers": ["myServer","anotherServer"]                 // optional, list of servers to scrobble tracks from
+    }
+  }
+]

docs/configuration.md

@@ -0,0 +1,219 @@
+# General
+
+General configuration options. These must be set through environmental variables because they affect initial startup of
+the app. **These variables are also available to Docker containers.**
+
+| Environmental Variable | Required? |   Default    |                                        Description                                        |
+|----------------------------|-----------|--------------|-------------------------------------------------------------------------------------------|
+| `CONFIG_DIR`               |         No | `CWD/config` | Directory to look for all other configuration files                                       |
+| `LOG_PATH`                 |         No | `CWD/logs`   | If `false` no logs will be written. If `string` will be the directory logs are written to |
+| `PORT`                     |         No | 9078         | Port to run web server on                                                                 |
+
+**The app must have permission to write to `CONFIG_DIR` in order to store the current spotify access token.**
+
+
+# Sources and (Scrobble) Clients
+
+The app has two types of configurations:
+
+* **Sources** -- Where plays are parsed from
+* **Clients** -- Scrobble clients that plays are scrobbled to
+
+All configurations can be configured through:
+* environmental variables
+* individual **json** files for each source/client type found in the `CONFIG_DIR` directory IE `config/spotify.json`
+* or through the main `config.json` (also found in `CONFIG_DIR` directory) using the `clients` or `sources` property under `data`:
+
+```json5
+// in config.json
+{
+  //...
+  "sources": [
+    {
+      "name": "myConfig",
+      "type": "spotify",
+      "clients": [
+        "myMalojaClient"
+      ],
+      "data": {
+        "clientId": "anExample"
+        //...
+      }
+    }
+  ]
+}
+```
+
+See [config.json.example](../config/config.json.example) for a short example of this or check out [the kitchen sink example](kitchensink.md).
+
+### ENV-Based or JSON-Based?
+
+multi-scrobbler can be configured differently depending on how you will use it. See which use-case fits you the best and then use that approach when setting up each configuration:
+
+#### ENV-Based (Single User)
+
+* You are the only person for whom the application is scrobbling
+* You may have many sources (Plex, Spotify, Tautulli...) but you only have one of each type of source
+* You have only one scrobble client
+* **Easier for small setups. Difficult for larger, multi-sourced setups (may want to switch to json)**
+* **Will not work for multi-user setups**
+
+#### JSON-Based (Multi User)
+
+* You are a single user but want to set up many sources
+* You want to use multi-scrobbler to scrobble for yourself and others IE family, friends, etc.
+* You may have many of each type of **Source** you want to scrobble from IE 2x Plex accounts, 3x Spotify accounts, 1x
+  Funkwhale...
+* You have more than one scrobble **Client** you want to scrobble to IE multiple Maloja servers, one for each person
+* You want only some **Sources** to scrobble to some **Clients** IE Fred's Spotify account scrobbles to Fred's Maloja
+  server, but not Mary's Maloja server
+
+Note: While you may mix and match configuration approaches it is recommended to **only use ENV-based configs if you are
+doing everything in ENV-based configurations.**
+
+# Sources
+
+## [Spotify](https://www.spotify.com)
+
+To access your Spotify history you must [register an application](https://developer.spotify.com/dashboard) to get a
+Client ID/Secret. Make sure to also whitelist your redirect URI in the application settings.
+
+### ENV-Based
+
+| Environmental Variable     | Required? |            Default             |                    Description                     |
+|----------------------------|-----------|----------------------------------|----------------------------------------------------|
+| `SPOTIFY_CLIENT_ID`        | Yes       |                                  |                                                    |
+| `SPOTIFY_CLIENT_SECRET`    | Yes       |                                  |                                                    |
+| `SPOTIFY_ACCESS_TOKEN`     | No         |                                  | Must include either this token or client id/secret |
+| `SPOTIFY_REFRESH_TOKEN`    | No         |                                  | If using access token this is also recommended      |
+| `SPOTIFY_REDIRECT_URI`     | No         | `http://localhost:{PORT}/callback` | URI must end in `callback`                         |
+
+### JSON-Based
+
+See [`spotify.json.example`](../config/spotify.json.example)
+
+## [Plex](https://plex.tv)
+
+Check the [instructions](plex.md) on how to setup a [webhooks](https://support.plex.tv/articles/115002267687-webhooks) to scrobble your plays.
+
+### ENV-Based
+
+| Environmental Variable | Required | Default |                   Description                   |
+|------------------------|----------|---------|-------------------------------------------------|
+| `PLEX_USER`              |        No |         | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
+
+### JSON-Based
+
+See [`plex.json.example`](../config/plex.json.example)
+
+## [Tautulli](https://tautulli.com)
+
+Check the [instructions](plex.md) on how to setup a notification agent.
+
+### ENV-Based
+
+| Environmental Variable | Required | Default |                   Description                   |
+|------------------------|----------|---------|-------------------------------------------------|
+| `TAUTULLI_USER`              |        No |         | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
+
+### JSON-Based
+
+See [`tautulli.json.example`](../config/tautulli.json.example)
+
+## [Subsonic](http://www.subsonic.org/)
+
+Can use this source for any application that implements the [Subsonic API](http://www.subsonic.org/pages/api.jsp) (such as [Airsonic](https://airsonic.github.io/))
+
+**Known Issues:**
+* "Time played at" is somewhat inaccurate since the api only reports "played X minutes ago" so...
+  * All scrobble times are therefore "on the minute" and you may experience occasional duplicate scrobbles
+  * "played X minutes ago" sometimes is also not reported correctly
+* Multiple artists are reported as one value and cannot be separated
+* If using [Airsonic Advanced](https://github.com/airsonic-advanced/airsonic-advanced) the password used (under **Credentials**) must be **Decodable**  
+
+### ENV-Based
+
+| Environmental Variable     | Required? |            Default             |                    Description                     |
+|----------------------------|-----------|----------------------------------|----------------------------------------------------|
+| `SUBSONIC_USER`        | Yes       |                                  |                                                    |
+| `SUBSONIC_PASSWORD`    | Yes       |                                  |                                                    |
+| `SUBSONIC_URL`     | Yes         |                                  | Base url of your subsonic-api server |
+
+### JSON-Based
+
+See [`subsonic.json.example`](../config/subsonic.json.example)
+
+## [Jellyfin](https://jellyfin.org/)
+
+Must be using Jellyfin 10.7 or greater
+
+* Add the [Webhook Plugin](https://github.com/crobibero/jellyfin-plugin-webhook) repository to your plugins, then restart your server
+* In the Webhook settings:
+  * `Add Generic Destination`
+  * In the new `Generic` dropdown:
+    * Webhook Url: `http://localhost:9078/jellyfin`
+    * Notification Type: `Playback Progress`
+    * Item Type: `Songs`
+    * Check `Send All Properties`
+  * Save
+
+### ENV-Based
+
+| Environmental Variable | Required? | Default | Description                                                       |
+|------------------------|-----------|---------|-------------------------------------------------------------------|
+| `JELLYFIN_USER`        |           |         | Comma-separated list of usernames (from Jellyfin) to scrobble for |
+| `JELLYFIN_SERVER`      |           |         | Comma-separated list of Jellyfin server names to scrobble from    |
+
+### JSON-Based
+
+See [`jellyfin.json.example`](../config/jellyfin.json.example)
+
+## [Last.fm (Source)](https://www.last.fm)
+
+See the [Last.fm (Client)](#lastfm) setup for registration instructions.
+
+### ENV-Based
+
+No support for ENV based for Last.fm as a client (only source)
+
+### JSON-Based
+
+See [`lastfm.json.example`](../config/lastfm.json.example), change `configureAs` to `source`.
+
+# Clients
+
+## [Maloja](https://github.com/krateng/maloja)
+
+### ENV-Based
+
+| Environmental Variable | Required? | Default |          Description          |
+|----------------------------|-----------|---------|-------------------------------|
+| `MALOJA_URL`               | Yes       |         | Base URL of your installation |
+| `MALOJA_API_KEY`           | Yes       |         | Api Key                       |
+
+### JSON-Based
+
+See [`maloja.json.example`](../config/maloja.json.example)
+
+## [Last.fm](https://www.last.fm)
+
+[Register for an API account here.](https://www.last.fm/api/account/create)
+
+The Callback URL is actually specified by multi-scrobbler but to keep things consistent you should use
+```
+http://localhost:9078/lastfm/callback
+```
+or replace `localhost:9078` with your own base URL
+
+### ENV-Based
+
+| Environmental Variable | Required? | Default |          Description          |
+|----------------------------|-----------|---------|-------------------------------|
+| `LASTFM_API_KEY`           | Yes       |         | Api Key from your API Account |
+| `LASTFM_SECRET`            | Yes       |         | Shared secret from your API Account |
+| `LASTFM_REDIRECT_URI`      | No        | `http://localhost:{PORT}/lastfm/callback`        | Url to use for authentication. Must include `lastfm/callback` somewhere in it |
+| `LASTFM_SESSION`           | No        |         | Session id. Will be generated by authentication flow if not provided.                       |
+
+### JSON-Based
+
+See [`lastfm.json.example`](../config/lastfm.json.example)

docs/kitchensink.md

@@ -0,0 +1,238 @@
+# Example Config using all Possible Features
+
+Scenario: 
+
+* You want to scrobble plays for yourself (Foxx), Fred, and Mary
+* Each person has their own Maloja server
+* Each person has their own Spotify account
+* You have your own Airsonic (subsonic) server you to scrobble from
+* Mary has her own Last.fm account she also wants to scrobble to
+* Fred has his own Spotify application and provides you with just his access and refresh token because he doesn't trust you (wtf Fred)
+* Fred has a Plex server and wants to scrobble everything he plays
+* Mary uses Fred's Plex server but only wants to scrobble her plays from the `podcast` library
+* The three of you have a shared library on Plex called `party` that you only play when you are hanging out. You want plays from that library to be scrobbled to everyone's servers.
+
+### All-in-one Config
+
+Using just one config file located at `CONFIG_DIR/config.json`:
+
+```json5
+{
+  "sourceDefaults": {
+    "maxPollRetries": 0,          // optional, default # of automatic polling restarts on error. can be overridden by property in individual config
+    "maxRequestRetries": 1,       // optional, default # of http request retries a source can make before error is thrown. can be overridden by property in individual config
+    "retryMultiplier": 1.5,       // optional, default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying). can be overridden by property in individual config
+  },
+  "clientDefaults": {
+    "maxRequestRetries": 1,       // optional, default # of http request retries a client can make before error is thrown. can be overridden by property in individual config
+    "retryMultiplier": 1.5,       // optional, default retry delay multiplier (retry attempt * multiplier = # of seconds to wait before retrying). can be overridden by property in individual config
+  },
+  "sources": [
+    {
+      "type": "spotify",
+      "name": "foxxSpot",
+      "clients": ["foxxMaloja"],
+      "data": {
+        "clientId": "foxxSpotifyAppId", 
+        "clientSecret": "foxxSpotifyAppSecret",
+        "maxRequestRetries": 2,  // override default max retries because spotify can...spotty
+      }
+    },
+    {
+      "type": "spotify",
+      "name": "marySpot",
+      "clients": ["maryMaloja"],
+      "data": {
+        "clientId": "foxxSpotifyAppId", // only need one application, it can be used by all users of this multi-scrobbler instance 
+        "clientSecret": "foxxSpotifyAppSecret",
+      }
+    },
+    {
+      "type": "spotify",
+      "name": "fredSpot",
+      "clients": ["fredMaloja"],
+      "data": {
+        "accessToken": "fredsToken",
+        "refreshToken": "fredsRefreshToken",
+        "interval": 120, // he also wants a slower check interval because his application already has heavy api usage
+      }
+    },
+    {
+      "type": "plex",
+      "name": "fredPlex",
+      "clients": ["fredMaloja"],
+      "data": {
+        "user": ["fred@email.com"]
+      }
+    },
+    {
+      "type": "plex",
+      "name": "maryPlex",
+      "clients": ["maryMaloja"],
+      "data": {
+        "user": ["mary@email.com"], // still need to specify mary as user so not all users who play from 'podcasts' get scrobbled
+        "libraries": ["podcasts"]
+      }
+    },
+    {
+      "type": "plex",
+      "name": "partyPlex",
+      // omitting clients (or making it empty) will make this Source scrobble to all Clients
+      "data": {
+        "libraries": ["party"],
+      }
+    },
+    {
+      "type": "subsonic",
+      "name": "foxxAirsonic",
+      "clients": ["foxxMaloja"],
+      "data": {
+        "user": "foxx",
+        "password": "foxxPassword",
+        "url": "https://airsonic.foxx.example"
+      }
+    },
+  ],
+  "clients": [
+    {
+      "type": "maloja",
+      "name": "foxxMaloja",
+      "data": {
+        "url": "https://maloja.foxx.example",
+        "apiKey": "foxxApiKey"
+      }
+    },
+    {
+      "type": "maloja",
+      "name": "fredMaloja",
+      "data": {
+        "url": "https://maloja.fred.example",
+        "apiKey": "fredApiKey"
+      }
+    },
+    {
+      "type": "maloja",
+      "name": "maryMaloja",
+      "data": {
+        "url": "https://maloja.mary.example",
+        "apiKey": "maryApiKey"
+      }
+    },
+    {
+      "type": "lastfm",
+      "name": "maryLFM",
+      "data": {
+        "apiKey": "maryApiKey",
+        "secret": "marySecret",
+      }
+    }
+  ]
+}
+
+```
+
+### Separate JSON files
+
+In `CONFIG_DIR/spotify.json`:
+
+```json5
+[
+  {
+    // may omit 'type' property since app knows this is file is for spotify configs
+    "name": "foxxSpot",
+    "clients": ["foxxMaloja"],
+    "data": {
+      "clientId": "foxxSpotifyAppId",
+      "clientSecret": "foxxSpotifyAppSecret",
+    }
+  },
+  {
+    "name": "marySpot",
+    "clients": ["maryMaloja"],
+    "data": {
+      "clientId": "foxxSpotifyAppId",
+      "clientSecret": "foxxSpotifyAppSecret",
+    }
+  },
+  {
+    "name": "fredSpot",
+    "clients": ["fredMaloja"],
+    "data": {
+      "accessToken": "fredsToken",
+      "refreshToken": "fredsRefreshToken",
+      "interval": 120,
+    }
+  },
+]
+```
+
+In `CONFIG_DIR/plex.json`
+
+```json5
+[
+  {
+    "name": "fredPlex",
+    "clients": ["fredMaloja"],
+    "data": {
+      "user": ["fred@email.com"]
+    }
+  },
+  {
+    "name": "maryPlex",
+    "clients": ["maryMaloja"],
+    "data": {
+      "user": ["mary@email.com"],
+      "libraries": ["podcasts"]
+    }
+  },
+  {
+    "name": "partyPlex",
+    "data": {
+      "libraries": ["party"],
+    }
+  }
+]
+```
+
+In `CONFIG_DIR/maloja.json`:
+
+```json5
+[
+  {
+    "name": "foxxMaloja",
+    "data": {
+      "url": "https://maloja.foxx.example",
+      "apiKey": "foxxApiKey"
+    }
+  },
+  {
+    "name": "fredMaloja",
+    "data": {
+      "url": "https://maloja.fred.example",
+      "apiKey": "fredApiKey"
+    }
+  },
+  {
+    "name": "maryMaloja",
+    "data": {
+      "url": "https://maloja.mary.example",
+      "apiKey": "maryApiKey"
+    }
+  }
+]
+```
+
+In `CONFIG_DIR/lastfm.json`:
+
+```json5
+[
+  {
+    "name": "maryLFM",
+    "data": {
+      "apiKey": "maryApiKey",
+      "secret": "marySecret",
+    }
+  }
+]
+```
+

docs/plex.md

@@ -0,0 +1,59 @@
+Tracks played on [Plex](https://plex.tv/) can be scrobbled either by:
+ * A [Tautulli](https://tautulli.com/) notification agent with a webhook.
+ * Using Plex [Webhooks](https://support.plex.tv/articles/115002267687-webhooks) (restricted to Plex Pass users)
+
+# Using Tautulli
+
+## Create a new Notification Agent
+
+* Navigate to the **Notification Agents** page in **Settings**
+* Click **Add a new notification agent**
+* Select **Webhook**
+
+## Configure the Agent
+
+The below sections correspond with the tabs available in the notification agent configuration popup.
+
+### Configuration
+* Webhook URL -- `http://localhost:9078/tautulli` (substitute your domain if different than the default)
+* Webhook Method -- POST
+
+### Triggers
+
+Select **Watched**
+
+### Conditions
+
+Refer to [Tautulli's documentation](https://github.com/Tautulli/Tautulli-Wiki/wiki/Custom-Notification-Conditions) if you need help here. It may be a good idea to restrict notifications to only one library (if you have a Music library, for instance)
+
+**This app will only scrobble an item if `media_type` is a "track", which is the default for all music.**
+
+### Data
+
+Expand the **Watched** dropdown and add the following code block to the **JSON Data** text field:
+
+```
+{
+"artist_name": "{artist_name}",
+"track_name": "{track_name}",
+"track_artist": "{track_artist}",
+"album_name": "{album_name}",
+"media_type": "{media_type}",
+"title":  "{title}",
+"duration": "{duration}",
+"username": "{username}"
+}
+```
+
+**Click the Save button to finish.**
+
+Your agent is now configured and ready to scrobble.
+
+# Using Plex Webhooks
+
+* Navigate to your **Account/Settings** and find the **Webhooks** page
+* Click **Add Webhook**
+* URL -- `http://localhost:9078/plex` (substitute your domain if different than the default)
+* **Save Changes**
+
+Plex is now configured to scrobble.

index.js

@@ -1,316 +1,479 @@
-import fs from "fs";
-import {addAsync} from '@awaitjs/express';
+import {addAsync, Router} from '@awaitjs/express';
 import express from 'express';
+import bodyParser from 'body-parser';
+import multer from 'multer';
 import winston from 'winston';
+import dayjs from 'dayjs';
+import utc from 'dayjs/plugin/utc.js';
+import isBetween from 'dayjs/plugin/isBetween.js';
+import relativeTime from 'dayjs/plugin/relativeTime.js';
+import duration from 'dayjs/plugin/duration.js';
 import {Writable} from 'stream';
 import 'winston-daily-rotate-file';
-import {readJson, sleep, writeFile, buildTrackString} from "./utils.js";
-import SpotifyWebApi from "spotify-web-api-node";
-import MalojaScrobbler from "./clients/MalojaScrobbler.js";
+import {
+    buildTrackString,
+    capitalize,
+    labelledFormat,
+    longestString,
+    readJson, sleep,
+    truncateStringToLength
+} from "./utils.js";
+import Clients from './clients/ScrobbleClients.js';
+import ScrobbleSources from "./sources/ScrobbleSources.js";
+import {makeClientCheckMiddle, makeSourceCheckMiddle} from "./server/middleware.js";
+import TautulliSource from "./sources/TautulliSource.js";
+import PlexSource from "./sources/PlexSource.js";
+import JellyfinSource from "./sources/JellyfinSource.js";
 
-const {format, createLogger, transports} = winston;
-const {combine, printf, timestamp} = format;
+const storage = multer.memoryStorage()
+const upload = multer({storage: storage})
+
+dayjs.extend(utc)
+dayjs.extend(isBetween);
+dayjs.extend(relativeTime);
+dayjs.extend(duration);
+
+const {transports} = winston;
 
 let output = []
 const stream = new Writable()
 stream._write = (chunk, encoding, next) => {
-    output.unshift(chunk.toString());
-    output = output.slice(0, 51);
+    let formatString = chunk.toString().replace('\n', '<br />')
+    .replace(/(debug)/gi, '<span class="debug text-pink-400">$1</span>')
+    .replace(/(warn)/gi, '<span class="warn text-blue-400">$1</span>')
+    .replace(/(info)/gi, '<span class="info text-yellow-500">$1</span>')
+    .replace(/(error)/gi, '<span class="error text-red-400">$1</span>')
+    output.unshift(formatString);
+    output = output.slice(0, 101);
     next()
 }
-const streamTransport = new winston.transports.Stream({stream})
+const streamTransport = new winston.transports.Stream({
+    stream,
+})
 
-const myFormat = printf(({level, message, label = 'App', timestamp}) => {
-    return `${timestamp} [${label}] ${level}: ${message}`;
+const logConfig = {
+    level: process.env.LOG_LEVEL || 'info',
+    sort: 'descending',
+    limit: 50,
+}
+
+const availableLevels = ['info', 'debug'];
+const logPath = process.env.LOG_DIR || `${process.cwd()}/logs`;
+const port = process.env.PORT ?? 9078;
+const localUrl = `http://localhost:${port}`;
+
+const rotateTransport = new winston.transports.DailyRotateFile({
+    dirname: logPath,
+    createSymlink: true,
+    symlinkName: 'scrobble-current.log',
+    filename: 'scrobble-%DATE%.log',
+    datePattern: 'YYYY-MM-DD',
+    maxSize: '5m'
 });
 
-const logger = createLogger({
-    level: 'debug',
-    format: combine(
-        timestamp(),
-        myFormat
-    ),
-    transports: [
-        new transports.Console(),
-        streamTransport,
-    ]
-});
+const consoleTransport = new transports.Console();
 
-const scopes = ['user-read-recently-played', 'user-read-currently-playing'];
-const state = 'random';
-let lastTrackPlayedAt = new Date();
+const myTransports = [
+    consoleTransport,
+    streamTransport,
+];
+
+if (typeof logPath === 'string') {
+    myTransports.push(rotateTransport);
+}
+
+const loggerOptions = {
+    level: logConfig.level,
+    format: labelledFormat(),
+    transports: myTransports,
+};
+
+winston.loggers.add('default', loggerOptions);
+
+const logger = winston.loggers.get('default');
 
 const configDir = process.env.CONFIG_DIR || `${process.cwd()}/config`;
-const configLocation = process.env.CONFIG_PATH || `${configDir}/config.json`;
 
 const app = addAsync(express());
+const router = Router();
 
-try {
-    (async function () {
-
-        let spotifyAsyncFunc = null;
+app.use(router);
+app.use(bodyParser.json());
 
+(async function () {
+    try {
         // try to read a configuration file
         let config = {};
         try {
-            config = await readJson(configLocation);
+            config = await readJson(`${configDir}/config.json`, {throwOnNotFound: false});
         } catch (e) {
-            logger.warn('Could not read config file');
-            logger.error(e);
+            logger.warn('App config file exists but could not be parsed!');
         }
 
         // setup defaults for other configs and general config
         const {
-            logPath: logPathRaw = process.env.LOG_PATH || true,
-            interval = 60,
-            port = process.env.PORT ?? 9078,
-            spotify: spotifyConfigRaw = process.env.SPOTIFY_CONFIG_PATH || `${configDir}/spotify.json`,
-            clients = [],
+            spotify,
+            plex,
         } = config || {};
 
-        const localUrl = `http://localhost:${port}`;
+        /*
+        * setup clients
+        * */
+        const scrobbleClients = new Clients(configDir);
+        await scrobbleClients.buildClientsFromConfig();
+        if (scrobbleClients.clients.length === 0) {
+            logger.warn('No scrobble clients were configured!')
+        }
 
-        // first thing, if user wants to log to file set it up now
-        if (logPathRaw !== false) {
-            let logPath = `${process.cwd()}/logs`;
-            if (typeof logPathRaw === 'string') {
-                logPath = logPathRaw;
+        const scrobbleSources = new ScrobbleSources(localUrl, configDir);
+        let deprecatedConfigs = [];
+        if (spotify !== undefined) {
+            logger.warn(`DEPRECATED: Using 'spotify' top-level property in config.json will be removed in next major version (0.4). Please use 'sources' instead.`)
+            deprecatedConfigs.push({
+                type: 'spotify',
+                name: 'unnamed',
+                source: 'config.json (top level)',
+                mode: 'single',
+                data: spotify
+            });
+        }
+        if (plex !== undefined) {
+            logger.warn(`DEPRECATED: Using 'plex' top-level property in config.json will be removed in next major version (0.4). Please use 'sources' instead.`)
+            deprecatedConfigs.push({
+                type: 'plex',
+                name: 'unnamed',
+                source: 'config.json (top level)',
+                mode: 'single',
+                data: plex
+            });
+        }
+        await scrobbleSources.buildSourcesFromConfig(deprecatedConfigs);
+
+        const clientCheckMiddle = makeClientCheckMiddle(scrobbleClients);
+        const sourceCheckMiddle = makeSourceCheckMiddle(scrobbleSources);
+
+        // check ambiguous client/source types like this for now
+        const lastfmSources = scrobbleSources.getByType('lastfm');
+        const lastfmScrobbles = scrobbleClients.getByType('lastfm');
+
+        const scrobblerNames = lastfmScrobbles.map(x => x.name);
+        const nameColl = lastfmSources.filter(x => scrobblerNames.includes(x.name));
+        if(nameColl.length > 0) {
+            logger.warn(`Last.FM source and clients have same names [${nameColl.map(x => x.name).join(',')}] -- this may cause issues`);
+        }
+
+        app.getAsync('/', async function (req, res) {
+            let slicedLog = output.slice(0, logConfig.limit + 1);
+            if (logConfig.sort === 'ascending') {
+                slicedLog.reverse();
             }
-            logger.add(new winston.transports.DailyRotateFile({
-                level: 'info', // don't need to add a bunch of noise to files
-                dirname: logPath,
-                createSymlink: true,
-                symlinkName: 'scrobble-current.log',
-                filename: 'scrobble-%DATE%.log',
-                datePattern: 'YYYY-MM-DD',
-                maxSize: '5m'
-            }))
-        }
+            // TODO links for re-trying auth and variables for signalling it (and API recently played)
+            const sourceData = scrobbleSources.sources.map((x) => {
+                const {
+                    type,
+                    tracksDiscovered = 0,
+                    name,
+                    canPoll = false,
+                    polling = false,
+                    initialized = false,
+                    requiresAuth = false,
+                    requiresAuthInteraction = false,
+                    authed = false,
+                } = x;
+                const base = {
+                    type,
+                    display: capitalize(type),
+                    tracksDiscovered,
+                    name,
+                    canPoll,
+                    hasAuth: requiresAuth,
+                    hasAuthInteraction: requiresAuthInteraction,
+                };
+                if(!initialized) {
+                    base.status = 'Not Initialized';
+                } else if(requiresAuth && !authed) {
+                    base.status = requiresAuthInteraction ? 'Auth Interaction Required' : 'Authentication Failed Or Not Attempted'
+                } else if(canPoll) {
+                    base.status = polling ? 'Running' : 'Idle';
+                } else {
+                    base.status = tracksDiscovered > 0 ? 'Received Data' : 'Awaiting Data'
+                }
+                return base;
+            });
+            const clientData = scrobbleClients.clients.map((x) => {
+                const {
+                    type,
+                    tracksScrobbled = 0,
+                    name,
+                    initialized = false,
+                    requiresAuth = false,
+                    requiresAuthInteraction = false,
+                    authed = false,
+                } = x;
+                const base = {
+                    type,
+                    display: capitalize(type),
+                    tracksDiscovered: tracksScrobbled,
+                    name,
+                    hasAuth: requiresAuth,
+                };
+                if(!initialized) {
+                    base.status = 'Not Initialized';
+                } else if(requiresAuth && !authed) {
+                    base.status = requiresAuthInteraction ? 'Auth Interaction Required' : 'Authentication Failed Or Not Attempted'
+                } else {
+                    base.status = tracksScrobbled > 0 ? 'Received Data' : 'Awaiting Data';
+                }
+                return base;
+            })
+            res.render('status', {
+                sources: sourceData,
+                clients: clientData,
+                logs: {
+                    output: slicedLog,
+                    limit: [10, 20, 50, 100].map(x => `<a class="capitalize ${logConfig.limit === x ? 'font-bold no-underline pointer-events-none' : ''}" href="logs/settings/update?limit=${x}">${x}</a>`).join(' | '),
+                    sort: ['ascending', 'descending'].map(x => `<a class="capitalize ${logConfig.sort === x ? 'font-bold no-underline pointer-events-none' : ''}" href="logs/settings/update?sort=${x}">${x}</a>`).join(' | '),
+                    level: availableLevels.map(x => `<a class="capitalize ${logConfig.level === x ? 'font-bold no-underline pointer-events-none' : ''}" href="logs/settings/update?level=${x}">${x}</a>`).join(' | ')
+                }
+            });
+        })
 
-        if (interval < 15) {
-            console.warn('Interval should be above 30 seconds...😬');
-        }
-
-        let spotifyCreds = {};
-        try {
-            spotifyCreds = await readJson('./spotifyCreds.json');
-        } catch (e) {
-            logger.warn('Current spotify access token was not parsable or file does not exist (this could be normal)');
-        }
-
-        let spotifyConfig = spotifyConfigRaw;
-        if (typeof spotifyConfigRaw === 'string') {
-            try {
-                spotifyConfig = await readJson(spotifyConfigRaw);
-            } catch (e) {
-                logger.warn('Could not read spotify config file');
-                logger.error(e);
+        app.postAsync('/tautulli', async function (req, res) {
+            const payload = TautulliSource.formatPlayObj(req.body, true);
+            // try to get config name from payload
+            if (req.body.scrobblerConfig !== undefined) {
+                const source = scrobbleSources.getByName(req.body.scrobblerConfig);
+                if (source !== undefined) {
+                    if (source.type !== 'tautulli') {
+                        this.logger.warn(`Tautulli event specified a config name but the configured source was not a Tautulli type: ${req.body.scrobblerConfig}`);
+                        return res.send('OK');
+                    } else {
+                        await source.handle(payload, scrobbleClients);
+                        return res.send('OK');
+                    }
+                } else {
+                    this.logger.warn(`Tautulli event specified a config name but no configured source found: ${req.body.scrobblerConfig}`);
+                    return res.send('OK');
+                }
             }
-        }
-
-        const {
-            accessToken = process.env.SPOTIFY_ACCESS_TOKEN,
-            clientId = process.env.SPOTIFY_CLIENT_ID,
-            clientSecret = process.env.SPOTIFY_CLIENT_SECRET,
-            redirectUri = process.env.SPOTIFY_REDIRECT_URI,
-            refreshToken = process.env.SPOTIFY_REFRESH_TOKEN,
-        } = spotifyConfig;
-
-        const rdUri = redirectUri || `${localUrl}/callback`;
-
-
-        const {token = accessToken, refreshToken: rt = refreshToken} = spotifyCreds;
-
-        if (token === undefined) {
-            if (clientId === undefined) {
-                throw new Error('ClientId not defined');
+            // if none specified we'll iterate through all tautulli sources and hopefully the user has configured them with filters
+            const tSources = scrobbleSources.getByType('tautulli');
+            for (const source of tSources) {
+                await source.handle(payload, scrobbleClients);
             }
-            if (clientSecret === undefined) {
-                throw new Error('ClientSecret not defined');
-            }
-        }
 
-        const spotifyApi = new SpotifyWebApi({
-            clientId,
-            clientSecret,
-            accessToken: token,
-            redirectUri: rdUri,
-            refreshToken: rt,
-        });
-
-        const scrobbleClients = await createClients(clients, configDir);
-        if (scrobbleClients.length === 0) {
-            throw new Error('No scrobble clients were configured');
-        }
-
-
-        app.getAsync('/authSpotify', async function (req, res) {
-            logger.info('Redirecting to spotify authorization url');
-            res.redirect(spotifyApi.createAuthorizeURL(scopes, state));
-        });
-
-        app.postAsync('/pollSpotify', async function (req, res) {
-            spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients);
             res.send('OK');
         });
 
-        app.getAsync(`/callback`, async function (req, res, next) {
-            const {error, code} = req.query;
-            if (error === undefined) {
-                const tokenResponse = await spotifyApi.authorizationCodeGrant(code);
-                spotifyApi.setAccessToken(tokenResponse.body['access_token']);
-                spotifyApi.setRefreshToken(tokenResponse.body['refresh_token']);
-                await writeFile('spotifyCreds.json', JSON.stringify({
-                    token: tokenResponse.body['access_token'],
-                    refreshToken: tokenResponse.body['refresh_token']
-                }));
-                logger.info('Got auth code from callback!');
-                spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients);
-                return res.send('OK');
-            } else {
-                throw new Error('User denied oauth access');
+        app.postAsync('/plex', upload.any(), async function (req, res) {
+            const {
+                body: {
+                    payload
+                } = {}
+            } = req;
+            if (payload !== undefined) {
+                const playObj = PlexSource.formatPlayObj(JSON.parse(payload), true);
+
+                const pSources = scrobbleSources.getByType('plex');
+                for (const source of pSources) {
+                    await source.handle(playObj, scrobbleClients);
+                }
+            }
+            res.send('OK');
+        });
+
+        // webhook plugin sends json with context type text/utf-8 so we need to parse it differently
+        const jellyfinJsonParser = bodyParser.json({type: 'text/*'});
+        app.postAsync('/jellyfin', jellyfinJsonParser, async function (req, res) {
+            const playObj = JellyfinSource.formatPlayObj(req.body, true);
+            const pSources = scrobbleSources.getByType('jellyfin');
+            for (const source of pSources) {
+                await source.handle(playObj, scrobbleClients);
+            }
+            res.send('OK');
+        });
+
+        app.use('/client/auth', clientCheckMiddle);
+        app.getAsync('/client/auth', async function (req, res) {
+            const {
+                scrobbleClient,
+            } = req;
+
+            switch (scrobbleClient.type) {
+                case 'lastfm':
+                    res.redirect(scrobbleClient.api.getAuthUrl());
+                    break;
+                default:
+                    return res.status(400).send(`Specified client does not have auth implemented (${scrobbleClient.type})`);
             }
         });
 
-        if (token === undefined) {
-            logger.info('No access token found');
-            logger.info(`Open ${localUrl}/authSpotify to continue`);
-        } else {
-            spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients)
-            logger.info(`Server started at ${localUrl}`);
-        }
+        app.use('/source/auth', sourceCheckMiddle);
+        app.getAsync('/source/auth', async function (req, res) {
+            const {
+                scrobbleSource: source,
+                sourceName: name,
+            } = req;
 
-        const server = await app.listen(port)
-    }());
-} catch (e) {
-    logger.error('Exited with uncaught error');
-    logger.error(e);
-}
-
-const pollSpotify = async function (spotifyApi, interval = 60, clients = []) {
-    logger.info('Starting spotify polling', {label: 'Spotify'});
-    try {
-        while (true) {
-            let data = {};
-            logger.debug('Refreshing recently played', {label: 'Spotify'})
-            try {
-                data = await spotifyApi.getMyRecentlyPlayedTracks({
-                    limit: 20
-                });
-            } catch (e) {
-                if (e.statusCode === 401) {
-                    logger.info('Access token was not valid, attempting to refresh', {label: 'Spotify'});
-                    const tokenResponse = await spotifyApi.refreshAccessToken();
-                    const {body: {
-                        access_token,
-                        // spotify may return a new refresh token
-                        // if it doesn't then continue to use the last refresh token we received
-                        refresh_token = spotifyApi.getRefreshToken(),
-                    } = {}} = tokenResponse;
-                    spotifyApi.setAccessToken(access_token);
-                    await writeFile('spotifyCreds.json', JSON.stringify({
-                        token: access_token,
-                        refreshToken: refresh_token,
-                    }));
-                    data = await spotifyApi.getMyRecentlyPlayedTracks({
-                        limit: 20
-                    });
-                } else {
-                    throw e;
-                }
-            }
-            let newLastPLayedAt = undefined;
-            const now = new Date();
-            for (const playObj of data.body.items) {
-                const {track: {name: trackName, duration_ms }, played_at} = playObj;
-                const playDate = new Date(played_at);
-                // compare play time to most recent track played_at scrobble
-                if (playDate.getTime() > lastTrackPlayedAt.getTime()) {
-                    logger.info(`New Track => ${buildTrackString(playObj)}`, {label: 'Spotify'});
-                    // so we always get just the most recent played_at
-                    if (newLastPLayedAt === undefined) {
-                        newLastPLayedAt = playDate;
+            switch (source.type) {
+                case 'spotify':
+                    if (source.spotifyApi === undefined) {
+                        res.status(400).send('Spotify configuration is not valid');
+                    } else {
+                        logger.info('Redirecting to spotify authorization url');
+                        res.redirect(source.createAuthUrl());
                     }
-                    const closeToInterval = Math.abs(now.getTime() - playDate.getTime()) / 1000 < 5;
-                    if (closeToInterval) {
-                        // because the interval check was so close to the play date we are going to delay client calls for a few secs
-                        // this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
-                        // additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
-                        logger.info('Track is close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first', {label: 'Spotify'});
-                        await sleep(10 * 1000);
-                    }
-                    for (const client of clients) {
-                        if (closeToInterval || client.scrobblesLastCheckedAt().getTime() < now.getTime()) {
-                            await client.refreshScrobbles();
-                        }
-                        if (!client.alreadyScrobbled(trackName, playDate, duration_ms / 1000)) {
-                            await client.scrobble(playObj);
-                        }
-                    }
-                } else {
                     break;
+                case 'lastfm':
+                    res.redirect(source.api.getAuthUrl());
+                    break;
+                default:
+                    return res.status(400).send(`Specified source does not have auth implemented (${source.type})`);
+            }
+        });
+
+        app.use('/poll', sourceCheckMiddle);
+        app.getAsync('/poll', async function (req, res) {
+            const {
+                scrobbleSource: source,
+            } = req;
+
+            if (!source.canPoll) {
+                return res.status(400).send(`Specified source cannot poll (${source.type})`);
+            }
+
+            source.poll(scrobbleClients);
+            res.send('OK');
+        });
+
+        app.use('/recent', sourceCheckMiddle);
+        app.getAsync('/recent', async function (req, res) {
+            const {
+                scrobbleSource: source,
+            } = req;
+            if (!source.canPoll) {
+                return res.status(400).send(`Specified source cannot retrieve recent plays (${source.type})`);
+            }
+
+            const result = await source.getRecentlyPlayed({formatted: true});
+            const artistTruncFunc = truncateStringToLength(Math.min(40, longestString(result.map(x => x.data.artists.join(' / ')).flat())));
+            const trackLength = longestString(result.map(x => x.data.track))
+            const plays = result.map((x) => {
+                const {
+                    meta: {
+                        url: {
+                            web
+                        } = {}
+                    } = {}
+                } = x;
+                const buildOpts = {
+                    include: ['time', 'timeFromNow', 'track', 'artist'],
+                    transformers: {
+                        artists: a => artistTruncFunc(a.join(' / ')).padEnd(33),
+                        track: t => t.padEnd(trackLength)
+                    }
                 }
-                if (newLastPLayedAt !== undefined) {
-                    lastTrackPlayedAt = newLastPLayedAt;
+                if (web !== undefined) {
+                    buildOpts.transformers.track = t => `<a href="${web}">${t}</a>${''.padEnd(Math.max(trackLength - t.length, 0))}`;
+                }
+                return buildTrackString(x, buildOpts);
+            });
+            res.render('recent', {plays, name: source.name, sourceType: source.type});
+        });
+
+        app.getAsync('/logs/settings/update', async function (req, res) {
+            const e = req.query;
+            for (const [setting, val] of Object.entries(req.query)) {
+                switch (setting) {
+                    case 'limit':
+                        logConfig.limit = Number.parseInt(val);
+                        break;
+                    case 'sort':
+                        logConfig.sort = val;
+                        break;
+                    case 'level':
+                        logConfig.level = val;
+                        for (const [key, logger] of winston.loggers.loggers) {
+                            logger.level = val;
+                        }
+                        break;
                 }
             }
-            // sleep for interval
-            logger.debug(`Sleeping for interval (${interval}s)`, {label: 'Spotify'});
-            await sleep(interval * 1000);
-        }
-    } catch (e) {
-        logger.error('Error occurred while in spotify polling loop', {label: 'Spotify'});
-        logger.error(e, {label: 'Spotify'});
-    }
-};
+            res.send('OK');
+        });
 
-
-const createClients = async function (clientConfigs = [], configDir = '.') {
-    const clients = [];
-    for (const config of clientConfigs) {
-        const dataType = typeof config;
-        if (!['object', 'string'].includes(dataType)) {
-            throw new Error('All client configs must be objects or strings');
-        }
-        const clientType = dataType === 'string' ? config : config.type;
-        switch (clientType) {
-            case 'maloja':
-                let data = `${configDir}/maloja.json`;
-                if (dataType === 'object') {
-                    const {data: dataProp = process.env.MALOJA_CONFIG_PATH || `${configDir}/maloja.json`} = config;
-                    data = dataProp;
-                }
-                let malojaConfig;
-                if (typeof data === 'string') {
-                    try {
-                        malojaConfig = await readJson(data);
-                    } catch (e) {
-                        logger.warn('Maloja config was not parsable or file does not exist');
-                    }
-                } else {
-                    malojaConfig = data;
-                }
+        app.getAsync(/.*callback$/, async function (req, res) {
+            const {
+                query: {
+                    state
+                } = {}
+            } = req;
+            if (req.url.includes('lastfm')) {
                 const {
-                    url = process.env.MALOJA_URL,
-                    apiKey = process.env.MALOJA_API_KEY
-                } = malojaConfig;
-                if (url === undefined && apiKey === undefined) {
-                    // the user probably didn't set anything up for this client at all, don't log
-                    continue;
+                    query: {
+                        token
+                    } = {}
+                } = req;
+                let entity = scrobbleClients.getByName(state);
+                if(entity === undefined) {
+                    entity = scrobbleSources.getByName(state);
                 }
-                if (url === undefined) {
-                    logger.warn('Maloja url not found in config');
-                    continue;
+                try {
+                    await entity.api.authenticate(token);
+                    await entity.initialize();
+                    return res.send('OK');
+                } catch (e) {
+                    return res.send(e.message);
                 }
-                if (apiKey === undefined) {
-                    logger.warn('Maloja api key not found in config');
-                    continue;
+            } else {
+                logger.info('Received auth code callback from Spotify', {label: 'Spotify'});
+                const source = scrobbleSources.getByName(state);
+                const tokenResult = await source.handleAuthCodeCallback(req.query);
+                let responseContent = 'OK';
+                if (tokenResult === true) {
+                    source.poll(scrobbleClients);
+                } else {
+                    responseContent = tokenResult;
                 }
-                clients.push(new MalojaScrobbler(logger, malojaConfig));
-                break;
-            default:
-                break;
+                return res.send(responseContent);
+            }
+        });
+
+        let anyNotReady = false;
+        for (const source of scrobbleSources.sources.filter(x => x.canPoll === true)) {
+            await sleep(1500); // stagger polling by 1.5 seconds so that log messages for each source don't get mixed up
+            switch (source.type) {
+                case 'spotify':
+                    if (source.spotifyApi !== undefined) {
+                        if (source.spotifyApi.getAccessToken() === undefined) {
+                            anyNotReady = true;
+                        } else {
+                            source.poll(scrobbleClients);
+                        }
+                    }
+                    break;
+                case 'lastfm':
+                    if(source.initialized === true) {
+                        source.poll(scrobbleClients);
+                    }
+                    break;
+                default:
+                    if (source.poll !== undefined) {
+                        source.poll(scrobbleClients);
+                    }
+            }
         }
+        if (anyNotReady) {
+            logger.info(`Some sources are not ready, open ${localUrl} to continue`);
+        }
+
+        app.set('views', './views');
+        app.set('view engine', 'ejs');
+        logger.info(`Server started at ${localUrl}`);
+        const server = await app.listen(port)
+    } catch (e) {
+        logger.error('Exited with uncaught error');
+        logger.error(e);
     }
-    return clients;
-}
+}());
+

package.json

@@ -1,7 +1,7 @@
 {
-  "name": "maloja-spotify-scrobbler",
+  "name": "multi-scrobbler",
   "version": "0.1.0",
-  "description": "",
+  "description": "scrobble plays from multiple sources to multiple clients",
   "type": "module",
   "main": "index.js",
   "scripts": {
@@ -13,19 +13,24 @@
   },
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/FoxxMD/maloja-spotify-scrobbler.git"
+    "url": "git+https://github.com/FoxxMD/multi-scrobbler.git"
   },
   "author": "",
   "license": "MIT",
   "bugs": {
-    "url": "https://github.com/FoxxMD/maloja-spotify-scrobbler/issues"
+    "url": "https://github.com/FoxxMD/multi-scrobbler/issues"
   },
-  "homepage": "https://github.com/FoxxMD/maloja-spotify-scrobbler#readme",
+  "homepage": "https://github.com/FoxxMD/multi-scrobbler#readme",
   "dependencies": {
     "@awaitjs/express": "^0.6.3",
-    "date-fns": "^2.16.1",
+    "body-parser": "^1.19.0",
+    "dayjs": "^1.10.4",
+    "ejs": "^3.1.6",
     "express": "^4.17.1",
-    "spotify-web-api-node": "^5.0.0",
+    "lastfm-node-client": "^2.2.0",
+    "multer": "^1.4.2",
+    "safe-stable-stringify": "^1.1.1",
+    "spotify-web-api-node": "^5.0.2",
     "superagent": "^6.1.0",
     "winston": "^3.3.3",
     "winston-daily-rotate-file": "^4.5.0"

server/middleware.js

@@ -0,0 +1,43 @@
+export const makeSourceCheckMiddle = sources => (req, res, next) => {
+    const {
+        query: {
+            name,
+            type
+        } = {}
+    } = req;
+
+    if (name === undefined) {
+        return res.status(404).send('Source name must be defined');
+    }
+
+    const source = sources.getByNameAndType(name, type);
+
+    if (source === undefined) {
+        return res.status(404).send(`No source with the name [${name}] and type [${type}`);
+    }
+
+    req.sourceName = name;
+    req.scrobbleSource = source;
+    next();
+}
+
+export const makeClientCheckMiddle = clients => (req, res, next) => {
+    const {
+        query: {
+            name
+        } = {}
+    } = req;
+
+    if (name === undefined) {
+        return res.status(404).send('Client name must be defined');
+    }
+
+    const client = clients.getByName(name);
+
+    if (client === undefined) {
+        return res.status(404).send(`No client with the name: ${name}`);
+    }
+
+    req.scrobbleClient = client;
+    next();
+}

sources/AbstractSource.js

@@ -0,0 +1,201 @@
+import dayjs from "dayjs";
+import {buildTrackString, capitalize, createLabelledLogger, sleep} from "../utils.js";
+
+export default class AbstractSource {
+
+    name;
+    type;
+    identifier;
+
+    config;
+    clients;
+    logger;
+    instantiatedAt;
+    initialized = false;
+    requiresAuth = false;
+    requiresAuthInteraction = false;
+    authed = false;
+
+    canPoll = false;
+    polling = false;
+    pollRetries = 0;
+    tracksDiscovered = 0;
+
+    constructor(type, name, config = {}, clients = []) {
+        this.type = type;
+        this.name = name;
+        this.identifier = `Source - ${capitalize(this.type)} - ${name}`;
+        this.logger = createLabelledLogger(this.identifier, this.identifier);
+        this.config = config;
+        this.clients = clients;
+        this.instantiatedAt = dayjs();
+    }
+
+    // default init function, should be overridden if init stage is required
+    initialize = async () => {
+        this.initialized = true;
+        return this.initialized;
+    }
+
+    // default init function, should be overridden if auth stage is required
+    testAuth = async () => {
+        return this.authed;
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        return [];
+    }
+
+    // by default if the track was recently played it is valid
+    // this is useful for sources where the track doesn't have complete information like Subsonic
+    // TODO make this more descriptive? or move it elsewhere
+    recentlyPlayedTrackIsValid = (playObj) => {
+        return true;
+    }
+
+    poll = async (allClients) => {
+        await this.startPolling(allClients);
+    }
+
+    startPolling = async (allClients) => {
+        if(this.requiresAuthInteraction && !this.authed) {
+            this.logger.error('Cannot start polling because user interaction is required for authentication');
+            return;
+        }
+        // reset poll attempts if already previously run
+        this.pollRetries = 0;
+
+        const {
+            maxPollRetries = 0,
+            retryMultiplier = 1.5,
+        } = this.config;
+
+        // can't have negative retries!
+        const maxRetries = Math.max(0, maxPollRetries);
+
+        while (this.pollRetries <= maxRetries) {
+            try {
+                await this.doPolling(allClients);
+            } catch (e) {
+                if (this.pollRetries < maxRetries) {
+                    const delayFor = (this.pollRetries + 1) * retryMultiplier;
+                    this.logger.info(`Poll reties (${this.pollRetries}) less than max poll retries (${maxRetries}), restarting polling after ${delayFor} second delay...`);
+                    await sleep((delayFor) * 1000);
+                } else {
+                    this.logger.warn(`Poll retries (${this.pollRetries}) equal to max poll retries (${maxRetries}), stopping polling!`);
+                }
+                this.pollRetries++;
+            }
+        }
+    }
+
+    /**
+     * @param {ScrobbleClients} allClients
+     */
+    doPolling = async (allClients) => {
+        if (this.polling === true) {
+            return;
+        }
+        this.logger.info('Polling started');
+        let lastTrackPlayedAt = this.instantiatedAt;
+        let checkCount = 0;
+        let checksOverThreshold = 0;
+        try {
+            this.polling = true;
+            while (true) {
+                if(this.polling === false) {
+                    this.logger.info('Stopped polling due to user input');
+                    break;
+                }
+                let playObjs = [];
+                this.logger.debug('Refreshing recently played')
+                playObjs = await this.getRecentlyPlayed({formatted: true});
+                checkCount++;
+                let newTracksFound = false;
+                let closeToInterval = false;
+                const now = dayjs();
+
+                const playInfo = playObjs.reduce((acc, playObj) => {
+                    if(this.recentlyPlayedTrackIsValid(playObj)) {
+                        const {data: {playDate} = {}} = playObj;
+                        if (playDate.unix() > lastTrackPlayedAt.unix()) {
+                            newTracksFound = true;
+                            this.logger.info(`New Track => ${buildTrackString(playObj)}`);
+
+                            if (closeToInterval === false) {
+                                closeToInterval = Math.abs(now.unix() - playDate.unix()) < 5;
+                            }
+
+                            return {
+                                plays: [...acc.plays, {...playObj, meta: {...playObj.meta, newFromSource: true}}],
+                                lastTrackPlayedAt: playDate
+                            }
+                        }
+                        return {
+                            ...acc,
+                            plays: [...acc.plays, playObj]
+                        }
+                    }
+                    return acc;
+                }, {plays: [], lastTrackPlayedAt});
+                playObjs = playInfo.plays;
+                lastTrackPlayedAt = playInfo.lastTrackPlayedAt;
+
+                if (closeToInterval) {
+                    // because the interval check was so close to the play date we are going to delay client calls for a few secs
+                    // this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
+                    // additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
+                    this.logger.info('Track is close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first');
+                    await sleep(10 * 1000);
+                }
+
+                if (newTracksFound === false) {
+                    if (playObjs.length === 0) {
+                        this.logger.debug(`No new tracks found and no tracks returned from API`);
+                    } else {
+                        this.logger.debug(`No new tracks found. Newest track returned was ${buildTrackString(playObjs.slice(-1)[0])}`);
+                    }
+                } else {
+                    checkCount = 0;
+                    checksOverThreshold = 0;
+                }
+
+                // use the source instantiation time or the last track play time to determine if we should refresh clients..
+                // we only need to refresh clients when the source has "newer" information otherwise we're just refreshing clients for no reason
+                const scrobbleResult = await allClients.scrobble(playObjs, {
+                    checkTime: lastTrackPlayedAt.add(2, 's'),
+                    forceRefresh: closeToInterval,
+                    scrobbleFrom: this.identifier,
+                    scrobbleTo: this.clients
+                });
+
+                if (scrobbleResult.length > 0) {
+                    checkCount = 0;
+                    this.tracksDiscovered += scrobbleResult.length;
+                }
+
+                const {interval = 30, checkActiveFor = 300, maxSleep = 300} = this.config;
+
+                let sleepTime = interval;
+                // don't need to do back off calc if interval is 5 minutes or greater since its already pretty light on API calls
+                // and don't want to back off if we just started the app
+                const activeThreshold = lastTrackPlayedAt.add(checkActiveFor, 's');
+                if (activeThreshold.isBefore(dayjs()) && sleepTime < 300) {
+                    checksOverThreshold++;
+                    const backoffMultiplier = Math.min(checksOverThreshold, 1000) * 1.5;
+                    sleepTime = Math.min(interval * backoffMultiplier, maxSleep);
+                }
+
+                // sleep for interval
+                this.logger.debug(`Sleeping for ${sleepTime}s`);
+                await sleep(sleepTime * 1000);
+
+            }
+        } catch (e) {
+            this.logger.error('Error occurred while polling');
+            this.logger.error(e);
+            this.polling = false;
+            throw e;
+        }
+    }
+}

sources/JellyfinSource.js

@@ -0,0 +1,165 @@
+import MemorySource from "./MemorySource.js";
+import dayjs from "dayjs";
+import {buildTrackString} from "../utils.js";
+
+
+export default class JellyfinSource extends MemorySource {
+    users;
+    servers;
+
+    constructor(name, config, clients, type = 'jellyfin') {
+        super(type, name, config, clients);
+        const {users, servers} = config
+
+        if (users === undefined || users === null) {
+            this.users = undefined;
+        } else {
+            if (!Array.isArray(users)) {
+                this.users = [users];
+            } else {
+                this.users = users;
+            }
+            this.users = this.users.map(x => x.toLocaleLowerCase())
+        }
+
+        if (servers === undefined || servers === null) {
+            this.servers = undefined;
+        } else {
+            if (!Array.isArray(servers)) {
+                this.servers = [servers];
+            } else {
+                this.servers = servers;
+            }
+            this.servers = this.servers.map(x => x.toLocaleLowerCase())
+        }
+
+        if (users === undefined && servers === undefined) {
+            this.logger.warn('Initializing, but with no filters! All tracks from all users on all servers will be scrobbled.');
+        } else {
+            this.logger.info(`Initializing with the following filters => Users: ${this.users === undefined ? 'N/A' : this.users.join(', ')} | Servers: ${this.servers === undefined ? 'N/A' : this.servers.join(', ')}`);
+        }
+        this.initialized = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            ServerId,
+            ServerName,
+            Username,
+            UserId,
+            NotificationType,
+            UtcTimestamp,
+            Album,
+            Artist,
+            Name,
+            RunTime,
+            ItemId,
+            ItemType,
+        } = obj;
+
+        const parsedRuntime = RunTime.split(':');
+        const dur = dayjs.duration({
+            hours: Number.parseInt(parsedRuntime[0]),
+            minutes: Number.parseInt(parsedRuntime[1]),
+            seconds: Number.parseInt(parsedRuntime[2])
+        });
+
+        return {
+            data: {
+                artists: [Artist],
+                album: Album,
+                track: Name,
+                duration: dur.as('seconds'),
+                playDate: dayjs(),
+            },
+            meta: {
+                event: NotificationType,
+                mediaType: ItemType,
+                sourceId: ItemId,
+                user: Username,
+                server: ServerName,
+                source: 'Jellyfin',
+                newFromSource,
+            }
+        }
+    }
+
+    isValidEvent = (playObj) => {
+        const {
+            meta: {
+                mediaType, event, user, server
+            },
+            data: {
+                artists,
+                track,
+            } = {}
+        } = playObj;
+
+        if (this.users !== undefined) {
+            if (user === undefined) {
+                this.logger.warn(`Config defined users but payload contained no user info${hint}`);
+            } else if (!this.users.includes(user.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because author was not an allowed user: ${user}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        if (event !== undefined && !['PlaybackProgress','PlaybackStarted'].includes(event)) {
+            this.logger.debug(`Will not scrobble event because it is not media.scrobble (${event})`, {
+                artists,
+                track
+            })
+            return false;
+        }
+
+        if (mediaType !== 'Audio') {
+            this.logger.debug(`Will not scrobble event because media type was not 'Audio' (${mediaType})`, {
+                artists,
+                track
+            });
+            return false;
+        }
+
+        if (this.servers !== undefined && !this.servers.includes(server.toLocaleLowerCase())) {
+            this.logger.debug(`Will not scrobble event because server was not on allowed list: ${server}`, {
+                artists,
+                track
+            })
+            return false;
+        }
+
+        return true;
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        return this.statefulRecentlyPlayed;
+    }
+
+    handle = async (playObj, allClients) => {
+        if (!this.isValidEvent(playObj)) {
+            return;
+        }
+
+        const newPlays = this.processRecentPlays([playObj]);
+
+        for(const p of newPlays) {
+            this.logger.info(`New Track => ${buildTrackString(p)}`);
+        }
+
+        if(newPlays.length > 0) {
+            const recent = await this.getRecentlyPlayed();
+            const newestPlay = recent[recent.length - 1];
+            try {
+                await allClients.scrobble(newPlays, {scrobbleTo: this.clients, scrobbleFrom: this.identifier, checkTime: newestPlay.data.playDate});
+                // only gets hit if we scrobbled ok
+                this.tracksDiscovered++;
+            } catch (e) {
+                this.logger.error('Encountered error while scrobbling')
+                this.logger.error(e)
+            }
+        }
+    }
+}

sources/LastfmSource.js

@@ -0,0 +1,79 @@
+import AbstractSource from "./AbstractSource.js";
+import LastfmApiClient from "../apis/LastfmApiClient.js";
+import {sortByPlayDate} from "../utils.js";
+
+export default class LastfmSource extends AbstractSource {
+
+    api;
+    requiresAuth = true;
+    requiresAuthInteraction = true;
+
+    constructor(name, config = {}, clients = []) {
+        super('lastfm', name, config, clients);
+        this.canPoll = true;
+        this.api = new LastfmApiClient(name, config);
+    }
+
+    static formatPlayObj(obj) {
+        return LastfmApiClient.formatPlayObj(obj);
+    }
+
+    initialize = async () => {
+        this.initialized = await this.api.initialize();
+        return this.initialized;
+    }
+
+    testAuth = async () => {
+        try {
+            this.authed = await this.api.testAuth();
+        } catch (e) {
+            this.logger.error('Could not successfully communicate with Last.fm API');
+            this.logger.error(e);
+            this.authed = false;
+        }
+        return this.authed;
+    }
+
+
+    getRecentlyPlayed = async(options = {}) => {
+        const {limit = 20, formatted = false} = options;
+        const resp = await this.api.callApi(client => client.userGetRecentTracks({user: this.api.user, limit, extended: true}));
+        const {
+            recenttracks: {
+                track: list = [],
+            }
+        } = resp;
+
+        return list.reduce((acc, x) => {
+            try {
+                const formatted = LastfmApiClient.formatPlayObj(x);
+                const {
+                    data: {
+                        track,
+                        playDate,
+                    },
+                    meta: {
+                        mbid,
+                        nowPlaying,
+                    }
+                } = formatted;
+                if(nowPlaying === true) {
+                    // if the track is "now playing" it doesn't get a timestamp so we can't determine when it started playing
+                    // and don't want to accidentally count the same track at different timestamps by artificially assigning it 'now' as a timestamp
+                    // so we'll just ignore it in the context of recent tracks since really we only want "tracks that have already finished being played" anyway
+                    this.logger.debug("Ignoring 'now playing' track returned from Last.fm client", {track, mbid});
+                    return acc;
+                } else if(playDate === undefined) {
+                    this.logger.warn(`Last.fm recently scrobbled track did not contain a timestamp, omitting from time frame check`, {track, mbid});
+                    return acc;
+                }
+                return acc.concat(formatted);
+            } catch (e) {
+                this.logger.warn('Failed to format Last.fm recently scrobbled track, omitting from time frame check', {error: e.message});
+                this.logger.debug('Full api response object:');
+                this.logger.debug(x);
+                return acc;
+            }
+        }, []).sort(sortByPlayDate);
+    }
+}

sources/MemorySource.js

@@ -0,0 +1,76 @@
+import AbstractSource from "./AbstractSource.js";
+import {playObjDataMatch, sortByPlayDate, buildTrackString} from "../utils.js";
+import dayjs from "dayjs";
+
+export default class MemorySource extends AbstractSource {
+    /*
+    * MemorySource uses its own state to maintain a list of recently played tracks and determine if a track if valid.
+    * This is necessary for any source that
+    *  * doesn't have its own source of truth for "recently played" or
+    *  * that does not return "started at" and "duration" timestamps for recent plays or
+    *  * where these timestamps don't have enough granularity (IE second accuracy)
+    * such as subsonic and jellyfin */
+
+    statefulRecentlyPlayed = [];
+    candidateRecentlyPlayed = [];
+
+    processRecentPlays = (plays) => {
+
+        let newStatefulPlays = [];
+        // first format new plays with locked play date
+        const lockedPlays = plays.map((p) => {
+                    const {data: {playDate, ...restData}, ...rest} = p;
+                    return {data: {...restData, playDate: dayjs()}, ...rest};
+        })
+        // if no candidates exist new plays are new candidates
+        if(this.candidateRecentlyPlayed.length === 0) {
+            this.candidateRecentlyPlayed = lockedPlays;
+        } else {
+            // otherwise determine new tracks (not found in prior candidates)
+            const newTracks = lockedPlays.filter(x => this.candidateRecentlyPlayed.every(y => !playObjDataMatch(y, x)));
+            // filter prior candidates based on new recently played
+            this.candidateRecentlyPlayed = this.candidateRecentlyPlayed.filter(x => lockedPlays.some(y => playObjDataMatch(x, y)));
+            // and then combine still playing with new tracks
+            this.candidateRecentlyPlayed = this.candidateRecentlyPlayed.concat(newTracks);
+            this.candidateRecentlyPlayed.sort(sortByPlayDate);
+
+            for(const candidate of this.candidateRecentlyPlayed) {
+                const {data: {playDate, track}} = candidate;
+                if(playDate.isBefore(dayjs().subtract(30, 's'))) {
+                    // a prior candidate has been playing for more than 30 seconds, time to check statefuls
+
+                    const matchingRecent = this.statefulRecentlyPlayed.find(x => playObjDataMatch(x, candidate));
+                    let stPrefix = `(Stateful Play) ${buildTrackString(candidate, {include: ['artist', 'track']})}`;
+                    if(matchingRecent === undefined) {
+                        this.logger.debug(`${stPrefix} added after being seen for 30 seconds and not matching any prior plays`);
+                        newStatefulPlays.push(candidate);
+                        this.statefulRecentlyPlayed.push(candidate);
+                    } else {
+                        const {data: { playDate, duration }} = candidate;
+                        const {data: { playDate: rplayDate }} = matchingRecent;
+                        if(!playDate.isSame(rplayDate)) {
+                            if(duration !== undefined) {
+                                if(playDate.isAfter(rplayDate.add(duration, 's'))) {
+                                    this.logger.debug(`${stPrefix} added after being seen for 30 seconds and having a different timestamp than a prior play`);
+                                    newStatefulPlays.push(candidate);
+                                    this.statefulRecentlyPlayed.push(candidate);
+                                }
+                            } else if(!playObjDataMatch(this.statefulRecentlyPlayed[0], candidate)) {
+                                // if most recent stateful play is not this track we'll add it
+                                this.logger.debug(`${stPrefix} added after being seen for 30 seconds. Matched other recent play but could not determine time frame due to missing duration. Allowed due to not being last played track.`);
+                                newStatefulPlays.push(candidate);
+                                this.statefulRecentlyPlayed.push(candidate);
+                            }
+                        }
+                    }
+                }
+            }
+            this.statefulRecentlyPlayed.sort(sortByPlayDate);
+        }
+        return newStatefulPlays;
+    }
+
+    recentlyPlayedTrackIsValid = (playObj) => {
+        return playObj.data.playDate.isBefore(dayjs().subtract(30, 's'));
+    }
+}

sources/PlexSource.js

@@ -0,0 +1,175 @@
+import dayjs from "dayjs";import LastFm from "lastfm-node-client";
+import LastfmScrobbler from '../clients/LastfmScrobbler.js';
+import {buildTrackString} from "../utils.js";
+import AbstractSource from "./AbstractSource.js";
+
+export default class PlexSource extends AbstractSource {
+    users;
+    libraries;
+    servers;
+
+    constructor(name, config, clients, type = 'plex') {
+        super(type, name, config, clients);
+        const {user, libraries, servers} = config
+
+        if (user === undefined || user === null) {
+            this.users = undefined;
+        } else {
+            if (!Array.isArray(user)) {
+                this.users = [user];
+            } else {
+                this.users = user;
+            }
+            this.users = this.users.map(x => x.toLocaleLowerCase())
+        }
+
+        if (libraries === undefined || libraries === null) {
+            this.libraries = undefined;
+        } else {
+            if (!Array.isArray(libraries)) {
+                this.libraries = [libraries];
+            } else {
+                this.libraries = libraries;
+            }
+            this.libraries = this.libraries.map(x => x.toLocaleLowerCase())
+        }
+
+        if (servers === undefined || servers === null) {
+            this.servers = undefined;
+        } else {
+            if (!Array.isArray(servers)) {
+                this.servers = [servers];
+            } else {
+                this.servers = servers;
+            }
+            this.servers = this.servers.map(x => x.toLocaleLowerCase())
+        }
+
+        if (user === undefined && libraries === undefined && servers === undefined) {
+            this.logger.warn('Initializing, but with no filters! All tracks from all users on all servers and libraries will be scrobbled.');
+        } else {
+            this.logger.info(`Initializing with the following filters => Users: ${this.users === undefined ? 'N/A' : this.users.join(', ')} | Libraries: ${this.libraries === undefined ? 'N/A' : this.libraries.join(', ')} | Servers: ${this.servers === undefined ? 'N/A' : this.servers.join(', ')}`);
+        }
+        this.initialized = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            event,
+            Account: {
+                title: user,
+            } = {},
+            Metadata: {
+                type,
+                title: track,
+                parentTitle: album,
+                grandparentTitle: artist,
+                librarySectionTitle: library
+            } = {},
+            Server: {
+                title: server
+            } = {},
+        } = obj;
+        return {
+            data: {
+                artists: [artist],
+                album,
+                track,
+                playDate: dayjs(),
+            },
+            meta: {
+                event,
+                mediaType: type,
+                user,
+                library,
+                server,
+                source: 'Plex',
+                newFromSource,
+            }
+        }
+    }
+
+    isValidEvent = (playObj) => {
+        const {
+            meta: {
+                mediaType, event, user, library, server
+            },
+            data: {
+                artists,
+                track,
+            } = {}
+        } = playObj;
+
+        const hint = this.type === 'tautulli' ? ' (Check notification agent json data configuration)' : '';
+
+        if (this.users !== undefined) {
+            if (user === undefined) {
+                this.logger.warn(`Config defined users but payload contained no user info${hint}`);
+            } else if (!this.users.includes(user.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because author was not an allowed user: ${user}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        if (event !== undefined && event !== 'media.scrobble') {
+            this.logger.debug(`Will not scrobble event because it is not media.scrobble (${event})`, {
+                artists,
+                track
+            })
+            return false;
+        }
+
+        if (mediaType !== 'track') {
+            this.logger.debug(`Will not scrobble event because media type was not a track (${mediaType})`, {
+                artists,
+                track
+            });
+            return false;
+        }
+
+        if (this.libraries !== undefined) {
+            if (library === undefined) {
+                this.logger.warn(`Config defined libraries but payload contained no library info${hint}`);
+            } else if (!this.libraries.includes(library.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because library was not on allowed list: ${library}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        if (this.servers !== undefined) {
+            if (server === undefined) {
+                this.logger.warn(`Config defined server but payload contained no server info${hint}`);
+            } else if (!this.servers.includes(server.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because server was not on allowed list: ${server}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        return true;
+    }
+
+    handle = async (playObj, allClients) => {
+        if (!this.isValidEvent(playObj)) {
+            return;
+        }
+
+        this.logger.info(`New Track => ${buildTrackString(playObj)}`);
+        try {
+            await allClients.scrobble(playObj, {scrobbleTo: this.clients, scrobbleFrom: this.identifier});
+            // only gets hit if we scrobbled ok
+            this.tracksDiscovered++;
+        } catch (e) {
+            this.logger.error('Encountered error while scrobbling')
+            this.logger.error(e)
+        }
+    }
+}

sources/ScrobbleSources.js

@@ -0,0 +1,327 @@
+import {createLabelledLogger, isValidConfigStructure, readJson} from "../utils.js";
+import SpotifySource from "./SpotifySource.js";
+import PlexSource from "./PlexSource.js";
+import TautulliSource from "./TautulliSource.js";
+import {SubsonicSource} from "./SubsonicSource.js";
+import JellyfinSource from "./JellyfinSource.js";
+import LastfmSource from "./LastfmSource.js";
+
+export default class ScrobbleSources {
+
+    sources = [];
+    logger;
+    configDir;
+    localUrl;
+
+    sourceTypes = ['spotify', 'plex', 'tautulli', 'subsonic', 'jellyfin', 'lastfm'];
+
+    constructor(localUrl, configDir = process.cwd()) {
+        this.configDir = configDir;
+        this.localUrl = localUrl;
+        this.logger = createLabelledLogger('sources', 'Sources');
+    }
+
+    getByName = (name) => {
+        return this.sources.find(x => x.name === name);
+    }
+
+    getByType = (type) => {
+        return this.sources.filter(x => x.type === type);
+    }
+
+    getByNameAndType = (name, type) => {
+        return this.sources.find(x => x.name === name && x.type === type);
+    }
+
+    buildSourcesFromConfig = async (additionalConfigs = []) => {
+        let configs = additionalConfigs;
+
+        let configFile;
+        try {
+            configFile = await readJson(`${this.configDir}/config.json`, {throwOnNotFound: false});
+        } catch (e) {
+            throw new Error('config.json could not be parsed');
+        }
+        let sourceDefaults = {};
+        if (configFile !== undefined) {
+            const {
+                sources: mainConfigSourcesConfigs = [],
+                sourceDefaults: sd = {},
+            } = configFile;
+            sourceDefaults = sd;
+            const validMainConfigs = mainConfigSourcesConfigs.reduce((acc, curr, i) => {
+                if(curr === null) {
+                    this.logger.error(`The source config entry at index ${i} in config.json is null but should be an object, will not parse`);
+                    return acc;
+                }
+                if(typeof curr !== 'object') {
+                    this.logger.error(`The source config entry at index ${i} in config.json should be an object, will not parse`);
+                    return acc;
+                }
+                return acc.concat(curr);
+            }, []);
+            for (const c of validMainConfigs) {
+                const {name = 'unnamed'} = c;
+                configs.push({...c,
+                    name,
+                    source: 'config.json',
+                    configureAs: 'source' // override user value
+                });
+            }
+        }
+
+        for (let sourceType of this.sourceTypes) {
+            let defaultConfigureAs = 'source';
+            // env builder for single user mode
+            switch (sourceType) {
+                case 'spotify':
+                    const s = {
+                        accessToken: process.env.SPOTIFY_ACCESS_TOKEN,
+                        clientId: process.env.SPOTIFY_CLIENT_ID,
+                        clientSecret: process.env.SPOTIFY_CLIENT_SECRET,
+                        redirectUri: process.env.SPOTIFY_REDIRECT_URI,
+                        refreshToken: process.env.SPOTIFY_REFRESH_TOKEN,
+                    };
+                    if (!Object.values(s).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'spotify',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: s
+                        })
+                    }
+                    break;
+                case 'tautulli':
+                    const t = {
+                        // support this for now
+                        user: process.env.TAUTULLI_USER || process.env.PLEX_USER
+                    };
+                    if (!Object.values(t).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'tautulli',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: t
+                        })
+                    }
+                    break;
+                case 'plex':
+                    const p = {
+                        user: process.env.PLEX_USER
+                    };
+                    if (!Object.values(p).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'plex',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: p
+                        })
+                    }
+                    break;
+                case 'subsonic':
+                    const sub = {
+                        user: process.env.SUBSONIC_USER,
+                        password: process.env.SUBSONIC_PASSWORD,
+                        url: process.env.SUBSONIC_URL,
+                    };
+                    if (!Object.values(sub).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'subsonic',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: sub
+                        })
+                    }
+                    break;
+                case 'jellyfin':
+                    const j = {
+                        user: process.env.JELLYFIN_USER,
+                        server: process.env.JELLYFIN_SERVER,
+                    };
+                    if (!Object.values(j).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'jellyfin',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: j
+                        })
+                    }
+                    break;
+                case 'lastfm':
+                    // sane default for lastfm is that user want to scrobble TO it, not FROM it -- this is also existing behavior
+                    defaultConfigureAs = 'client';
+                    break;
+                default:
+                    break;
+            }
+            let rawSourceConfigs;
+            try {
+                rawSourceConfigs = await readJson(`${this.configDir}/${sourceType}.json`, {throwOnNotFound: false});
+            } catch (e) {
+                this.logger.error(`${sourceType}.json config file could not be parsed`);
+                continue;
+            }
+            if (rawSourceConfigs !== undefined) {
+                let sourceConfigs = [];
+                if (Array.isArray(rawSourceConfigs)) {
+                    sourceConfigs = rawSourceConfigs;
+                } else if (rawSourceConfigs === null) {
+                    this.logger.error(`${sourceType}.json contained no data`);
+                    continue;
+                } else if (typeof rawSourceConfigs === 'object') {
+                    // backwards compatibility, assuming its single-user mode
+                    this.logger.warn(`DEPRECATED: Starting in 0.4 configurations in all [type].json files (${sourceType}.json) must be in an array.`);
+                    if (rawSourceConfigs.data === undefined) {
+                        sourceConfigs = [{data: rawSourceConfigs, mode: 'single', name: 'unnamed'}];
+                    } else {
+                        sourceConfigs = [rawSourceConfigs];
+                    }
+                } else {
+                    this.logger.error(`All top level data from ${sourceType}.json must be an array of objects, will not parse configs from file`);
+                    continue;
+                }
+                for (const [i,m] of sourceConfigs.entries()) {
+                    if(m === null) {
+                        this.logger.error(`The config entry at index ${i} from ${sourceType}.json is null`);
+                        continue;
+                    }
+                    if (typeof m !== 'object') {
+                        this.logger.error(`The config entry at index ${i} from ${sourceType}.json was not an object, skipping`, m);
+                        continue;
+                    }
+                    const {configureAs = defaultConfigureAs} = m;
+                    if(configureAs === 'source') {
+                        m.source = `${sourceType}.json`;
+                        m.type = sourceType;
+                        configs.push(m);
+                    }
+                }
+            }
+        }
+
+        // we have all possible configurations so we'll check they are minimally valid
+        const validConfigs = configs.reduce((acc, c) => {
+            const isValid = isValidConfigStructure(c, {type: true, data: true});
+            if (isValid !== true) {
+                this.logger.error(`Source config from ${c.source} with name [${c.name || 'unnamed'}] of type [${c.type || 'unknown'}] will not be used because it has structural errors: ${isValid.join(' | ')}`);
+                return acc;
+            }
+            return acc.concat(c);
+        }, []);
+
+        // finally! all configs are valid, structurally, and can now be passed to addClient
+        // do a last check that names (within each type) are unique and warn if not, but add anyways
+        const typeGroupedConfigs = validConfigs.reduce((acc, curr) => {
+            const {type} = curr;
+            const {[type]: t = []} = acc;
+            return {...acc, [type]: [...t, curr]};
+        }, {});
+        // only need to warn if dup names PER TYPE
+        for (const [type, typedConfigs] of Object.entries(typeGroupedConfigs)) {
+            const nameGroupedConfigs = typedConfigs.reduce((acc, curr) => {
+                const {name = 'unnamed'} = curr;
+                const {[name]: n = []} = acc;
+                return {...acc, [name]: [...n, curr]};
+            }, {});
+            for (const [name, namedConfigs] of Object.entries(nameGroupedConfigs)) {
+                let tempNamedConfigs = namedConfigs;
+                const hasDups = namedConfigs.length > 1;
+                if (hasDups) {
+                    const sources = namedConfigs.map(c => `Config object from ${c.source} of type [${c.type}]`);
+                    this.logger.warn(`Source configs have naming conflicts -- the following configs have the same name "${name}":\n\n${sources.join('\n')}\n`);
+                    if (name === 'unnamed') {
+                        this.logger.info('HINT: "unnamed" configs occur when using ENVs, if a multi-user mode config does not have a "name" property, or if a config is built in single-user mode');
+                    }
+                }
+                tempNamedConfigs = tempNamedConfigs.map(({name = 'unnamed', ...x}, i) => ({
+                    ...x,
+                    name: hasDups ? `${name}${i + 1}` : name
+                }));
+                for (const c of tempNamedConfigs) {
+                    try {
+                        await this.addSource(c, sourceDefaults);
+                    } catch(e) {
+                        this.logger.error(`Source ${c.name} of type ${c.type} was not added because of unrecoverable errors`);
+                        this.logger.error(e);
+                    }
+                }
+            }
+        }
+    }
+
+    addSource = async (clientConfig, defaults = {}) => {
+        const isValidConfig = isValidConfigStructure(clientConfig, {name: true, data: true, type: true});
+        if (isValidConfig !== true) {
+            throw new Error(`Config object from ${clientConfig.source || 'unknown'} with name [${clientConfig.name || 'unnamed'}] of type [${clientConfig.type || 'unknown'}] has errors: ${isValidConfig.join(' | ')}`)
+        }
+        const {type, name, clients = [], data: d = {}} = clientConfig;
+        // add defaults
+        const data = {...defaults, ...d};
+        this.logger.debug(`(${name}) Constructing ${type} source`);
+        let newSource;
+        switch (type) {
+            case 'spotify':
+                newSource = new SpotifySource(name, {
+                    ...data,
+                    localUrl: this.localUrl,
+                    configDir: this.configDir
+                }, clients);
+                break;
+            case 'plex':
+                newSource = await new PlexSource(name, data, clients);
+                break;
+            case 'tautulli':
+                newSource = await new TautulliSource(name, data, clients);
+                break;
+            case 'subsonic':
+                newSource = new SubsonicSource(name, data, clients);
+                break;
+            case 'jellyfin':
+                newSource = await new JellyfinSource(name, data, clients);
+                break;
+            case 'lastfm':
+                newSource = await new LastfmSource(name, {...data, configDir: this.configDir}, clients);
+                break;
+            default:
+                break;
+        }
+
+        if(newSource === undefined) {
+            // really shouldn't get here!
+            throw new Error(`Source of type ${type} was not recognized??`);
+        }
+        if(newSource.initialized === false) {
+            this.logger.debug(`(${name}) Attempting ${type} initialization...`);
+            if (await newSource.initialize() === false) {
+                this.logger.error(`(${name}) ${type} source failed to initialize. Source needs to be successfully initialized before activity capture can begin.`);
+                return;
+            } else {
+                this.logger.info(`(${name}) ${type} source initialized`);
+            }
+        } else {
+            this.logger.info(`(${name}) ${type} source initialized`);
+        }
+
+        if(newSource.requiresAuth && !newSource.authed) {
+            this.logger.debug(`(${name}) Checking ${type} source auth...`);
+            let success;
+            try {
+                success = await newSource.testAuth();
+            } catch (e) {
+                success = false;
+            }
+            if(!success) {
+                this.logger.warn(`(${name}) ${type} source auth failed.`);
+            } else {
+                this.logger.info(`(${name}) ${type} source auth OK`);
+            }
+        }
+
+        this.sources.push(newSource);
+    }
+}

sources/SpotifySource.js

@@ -0,0 +1,254 @@
+import dayjs from "dayjs";
+import {
+    readJson,
+    writeFile,
+    sortByPlayDate, sleep, parseRetryAfterSecsFromObj,
+} from "../utils.js";
+import SpotifyWebApi from "spotify-web-api-node";
+import AbstractSource from "./AbstractSource.js";
+
+const scopes = ['user-read-recently-played', 'user-read-currently-playing'];
+const state = 'random';
+
+export default class SpotifySource extends AbstractSource {
+
+    spotifyApi;
+    localUrl;
+    workingCredsPath;
+    configDir;
+
+    requiresAuth = true;
+    requiresAuthInteraction = true;
+
+    constructor(name, config = {}, clients = []) {
+        super('spotify', name, config, clients);
+        const {
+            localUrl,
+            configDir,
+            interval = 60,
+        } = config;
+
+        if (interval < 15) {
+            this.logger.warn('Interval should be above 30 seconds...😬');
+        }
+
+        this.config.interval = interval;
+
+        this.configDir = configDir;
+        this.workingCredsPath = `${configDir}/currentCreds-${name}.json`;
+        this.localUrl = localUrl;
+        this.canPoll = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            track: {
+                artists = [],
+                name,
+                id,
+                duration_ms,
+                album: {
+                    name: albumName,
+                } = {},
+                external_urls: {
+                    spotify,
+                } = {}
+            } = {},
+            played_at
+        } = obj;
+        //let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(',');
+        return {
+            data: {
+                artists: artists.map(x => x.name),
+                album: albumName,
+                track: name,
+                duration: duration_ms / 1000,
+                playDate: dayjs(played_at),
+            },
+            meta: {
+                trackLength: duration_ms / 1000,
+                source: 'Spotify',
+                sourceId: id,
+                newFromSource,
+                url: {
+                    web: spotify
+                }
+            }
+        }
+    }
+
+    buildSpotifyApi = async () => {
+
+        let spotifyCreds = {};
+        try {
+            spotifyCreds = await readJson(this.workingCredsPath, {throwOnNotFound: false});
+        } catch (e) {
+            this.logger.warn('Current spotify credentials file exists but could not be parsed', { path: this.workingCredsPath });
+        }
+
+        const {
+            accessToken,
+            clientId,
+            clientSecret,
+            redirectUri,
+            refreshToken,
+        } = this.config || {};
+
+        const rdUri = redirectUri || `${this.localUrl}/callback`;
+
+
+        const {token = accessToken, refreshToken: rt = refreshToken} = spotifyCreds || {};
+
+        const apiConfig = {
+            clientId,
+            clientSecret,
+            accessToken: token,
+            refreshToken: rt,
+        }
+
+        if (Object.values(apiConfig).every(x => x === undefined)) {
+            this.logger.info('No values found for Spotify configuration, skipping initialization');
+            return;
+        }
+        apiConfig.redirectUri = rdUri;
+
+        const validationErrors = [];
+
+        if (token === undefined) {
+            if (clientId === undefined) {
+                validationErrors.push('clientId must be defined when access token is not present');
+            }
+            if (clientSecret === undefined) {
+                validationErrors.push('clientSecret must be defined when access token is not present');
+            }
+            if (rdUri === undefined) {
+                validationErrors.push('redirectUri must be defined when access token is not present');
+            }
+            if (validationErrors.length !== 0) {
+                validationErrors.unshift('no access token is defined');
+            }
+        } else if (rt === undefined && (
+            clientId === undefined ||
+            clientSecret === undefined ||
+            rdUri === undefined
+        )) {
+            this.logger.warn('Access token is present but no refresh token is defined and remaining configuration is not sufficient to re-authorize. Without a refresh token API calls will fail after current token is expired.');
+        }
+
+        if (validationErrors.length !== 0) {
+            this.logger.warn(`Configuration was not valid:\*${validationErrors.join('\n')}`);
+            throw new Error('Failed to initialize a Spotify source');
+        }
+
+        this.spotifyApi = new SpotifyWebApi(apiConfig);
+    }
+
+    initialize = async () => {
+        if(this.spotifyApi === undefined) {
+            await this.buildSpotifyApi();
+        }
+        this.initialized = true;
+        return this.initialized;
+    }
+
+    testAuth = async () => {
+        try {
+            await this.callApi((api => api.getMe()));
+            this.authed = true;
+        } catch (e) {
+            this.logger.error('Could not successfully communicate with Spotify API');
+            this.authed = false;
+        }
+        return this.authed;
+    }
+
+    createAuthUrl = () => {
+        return this.spotifyApi.createAuthorizeURL(scopes, this.name);
+    }
+
+    handleAuthCodeCallback = async ({error, code}) => {
+        if (error === undefined) {
+            const tokenResponse = await this.spotifyApi.authorizationCodeGrant(code);
+            this.spotifyApi.setAccessToken(tokenResponse.body['access_token']);
+            this.spotifyApi.setRefreshToken(tokenResponse.body['refresh_token']);
+            await writeFile(this.workingCredsPath, JSON.stringify({
+                token: tokenResponse.body['access_token'],
+                refreshToken: tokenResponse.body['refresh_token']
+            }));
+            this.logger.info('Got token from code grant authorization!');
+            return true;
+        } else {
+            this.logger.warn('Callback contained an error! User may have denied access?')
+            this.logger.error(error);
+            return error;
+        }
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        const {limit = 20, formatted = false} = options;
+        const func = api => api.getMyRecentlyPlayedTracks({
+            limit
+        });
+        const result = await this.callApi(func);
+        if (formatted === true) {
+            return result.body.items.map(x => SpotifySource.formatPlayObj(x)).sort(sortByPlayDate);
+        }
+        return result;
+    }
+
+    callApi = async (func, retries = 0) => {
+        const {
+            maxRequestRetries = 1,
+            retryMultiplier = 2,
+        } = this.config;
+        try {
+            return await func(this.spotifyApi);
+        } catch (e) {
+            if (e.statusCode === 401) {
+                if (this.spotifyApi.getRefreshToken() === undefined) {
+                    throw new Error('Access token was not valid and no refresh token was present')
+                }
+                this.logger.debug('Access token was not valid, attempting to refresh');
+
+                const tokenResponse = await this.spotifyApi.refreshAccessToken();
+                const {
+                    body: {
+                        access_token,
+                        // spotify may return a new refresh token
+                        // if it doesn't then continue to use the last refresh token we received
+                        refresh_token = this.spotifyApi.getRefreshToken(),
+                    } = {}
+                } = tokenResponse;
+                this.spotifyApi.setAccessToken(access_token);
+                await writeFile(this.workingCredsPath, JSON.stringify({
+                    token: access_token,
+                    refreshToken: refresh_token,
+                }));
+                try {
+                    return await func(this.spotifyApi);
+                } catch (ee) {
+                    this.logger.error('Refreshing access token encountered an error');
+                    this.logger.error(ee, {label: 'Spotify'});
+                    throw ee;
+                }
+            } else if(maxRequestRetries > retries) {
+                const retryAfter = parseRetryAfterSecsFromObj(e) ?? (retryMultiplier * (retries + 1));
+                this.logger.warn(`Request failed but retries (${retries}) less than max (${maxRequestRetries}), retrying request after ${retryAfter} seconds...`);
+                await sleep(retryAfter * 1000);
+                return this.callApi(func, retries + 1);
+            } else {
+                this.logger.error(`Request failed on retry (${retries}) with no more retries permitted (max ${maxRequestRetries})`);
+                this.logger.error(e, {label: 'Spotify'});
+                throw e;
+            }
+        }
+    }
+
+    poll = async (allClients) => {
+        if (this.spotifyApi === undefined) {
+            this.logger.warn('Cannot poll spotify without valid credentials configuration')
+            return;
+        }
+        await this.startPolling(allClients);
+    }
+}

sources/SubsonicSource.js

@@ -0,0 +1,171 @@
+import AbstractSource from "./AbstractSource.js";
+import request from 'superagent';
+import crypto from 'crypto';
+import dayjs from "dayjs";
+import isSameOrAfter from "dayjs/plugin/isSameOrAfter.js";
+import {buildTrackString, parseRetryAfterSecsFromObj, sleep} from "../utils.js";
+import MemorySource from "./MemorySource.js";
+
+dayjs.extend(isSameOrAfter);
+
+export class SubsonicSource extends MemorySource {
+
+    requiresAuth = true;
+
+    constructor(name, config = {}, clients = []) {
+        // default to quick interval so we can get a decently accurate nowPlaying
+        const subsonicConfig = {interval: 10, maxSleep: 30, ...config};
+        super('subsonic', name, subsonicConfig, clients);
+
+        const {user, password, url} = this.config;
+
+        if (user === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'user' is not defined`);
+        }
+        if (password === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'password' is not defined`);
+        }
+        if (url === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'url' is not defined`);
+        }
+
+        this.canPoll = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            id,
+            title,
+            album,
+            artist,
+            duration, // seconds
+            minutesAgo,
+        } = obj;
+        return {
+            data: {
+                artists: [artist],
+                album,
+                track: title,
+                duration,
+                // subsonic doesn't return an exact datetime, only how many whole minutes ago it was played
+                // so we need to force the time to be 0 seconds always so that when we compare against scrobbles from client the time isn't off
+                playDate: minutesAgo === 0 ? dayjs().startOf('minute') : dayjs().startOf('minute').subtract(minutesAgo, 'minute'),
+            },
+            meta: {
+                trackLength: duration,
+                source: 'Subsonic',
+                sourceId: id,
+                newFromSource,
+            }
+        }
+    }
+
+    callApi = async (req, retries = 0) => {
+        const {
+            user,
+            password,
+            maxRequestRetries = 1,
+            retryMultiplier = 1.5
+        } = this.config;
+
+        const salt = await crypto.randomBytes(10).toString('hex');
+        const hash = crypto.createHash('md5').update(`${password}${salt}`).digest('hex')
+        req.query({
+            u: user,
+            t: hash,
+            s: salt,
+            v: '1.15.0',
+            c: `multi-scrobbler - ${this.name}`,
+            f: 'json'
+        });
+        try {
+            const resp = await req;
+            const {
+                body: {
+                    "subsonic-response": {
+                        status,
+                    },
+                    "subsonic-response": ssResp = {}
+                } = {}
+            } = resp;
+            if (status === 'failed') {
+                const err = new Error('Subsonic API returned an error');
+                err.response = resp;
+                throw  err;
+            }
+            return ssResp;
+        } catch (e) {
+            if(retries < maxRequestRetries) {
+                const retryAfter = parseRetryAfterSecsFromObj(e) ?? (retryMultiplier * (retries + 1));
+                this.logger.warn(`Request failed but retries (${retries}) less than max (${maxRequestRetries}), retrying request after ${retryAfter} seconds...`);
+                await sleep(retryAfter * 1000);
+                return await this.callApi(req, retries + 1)
+            }
+            const {
+                message,
+                response: {
+                    status,
+                    body: {
+                        "subsonic-response": {
+                            status: ssStatus,
+                            error: {
+                                code,
+                                message: ssMessage,
+                            } = {},
+                        } = {},
+                        "subsonic-response": ssResp
+                    } = {},
+                    text,
+                } = {},
+                response,
+            } = e;
+            let msg = response !== undefined ? `API Call failed: Server Response => ${ssMessage}` : `API Call failed: ${message}`;
+            const responseMeta = ssResp ?? text;
+            this.logger.error(msg, {status, response: responseMeta});
+            throw e;
+        }
+    }
+
+    initialize = async () => {
+        const {url} = this.config;
+        try {
+            await request.get(`${url}/`);
+            this.logger.info('Subsonic Connection: ok');
+            this.initialized = true;
+        } catch (e) {
+            if(e.status !== undefined && e.status !== 404) {
+                this.logger.info('Subsonic Connection: ok');
+                // we at least got a response!
+                this.initialized = true;
+            }
+        }
+
+        return this.initialized;
+    }
+
+    testAuth= async () => {
+        const {url} = this.config;
+        try {
+            await this.callApi(request.get(`${url}/rest/ping`));
+            this.authed = true;
+            this.logger.info('Subsonic API Status: ok');
+        } catch (e) {
+            this.authed = false;
+        }
+
+        return this.authed;
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        const {formatted = false} = options;
+        const {url} = this.config;
+        const resp = await this.callApi(request.get(`${url}/rest/getNowPlaying`));
+        const {
+            nowPlaying: {
+                entry = []
+            } = {}
+        } = resp;
+        this.processRecentPlays(entry.map(x => formatted ? SubsonicSource.formatPlayObj(x) : x));
+        return this.statefulRecentlyPlayed;
+    }
+}

sources/TautulliSource.js

@@ -0,0 +1,47 @@
+import dayjs from "dayjs";
+import PlexSource from "./PlexSource.js";
+
+export default class TautulliSource extends PlexSource {
+
+    constructor(name, config, clients) {
+        super(name, config, clients, 'tautulli');
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            artist_name,
+            track_name,
+            track_artist,
+            album_name,
+            media_type,
+            title,
+            library_name,
+            server,
+            duration,
+            username,
+        } = obj;
+        let artists = [artist_name];
+        if (track_artist !== undefined && track_artist !== artist_name) {
+            artists.push(track_artist);
+        }
+        return {
+            data: {
+                artists,
+                album: album_name,
+                track: track_name,
+                duration,
+                playDate: dayjs(),
+            },
+            meta: {
+                title,
+                library: library_name,
+                server,
+                mediaType: media_type,
+                user: username,
+                trackLength: duration,
+                source: 'Tautulli',
+                newFromSource,
+            }
+        }
+    }
+}

utils.js

@@ -1,18 +1,38 @@
 import fs, {promises, constants} from "fs";
+import dayjs from 'dayjs';
+import utc from 'dayjs/plugin/utc.js';
+import winston from "winston";
+import jsonStringify from 'safe-stable-stringify';
+import { TimeoutError, WebapiError } from "spotify-web-api-node/src/response-error.js";
+import { Response } from 'superagent';
 
-export async function readJson(path) {
-    await promises.access(path, constants.R_OK);
-    const data = await promises.readFile(path);
-    return JSON.parse(data);
+const {format} = winston;
+const {combine, printf, timestamp, label, splat, errors} = format;
 
-    // return new Promise((resolve, reject) => {
-    //     fs.readFile(path, 'utf8', function (err, data) {
-    //         if (err) {
-    //             reject(err);
-    //         }
-    //         resolve(JSON.parse(data));
-    //     });
-    // });
+dayjs.extend(utc);
+
+export async function readJson(path, {logErrors = true, throwOnNotFound = true} = {}) {
+    try {
+        await promises.access(path, constants.R_OK);
+        const data = await promises.readFile(path);
+        return JSON.parse(data);
+    } catch (e) {
+        const {code} = e;
+        if (code === 'ENOENT') {
+            if (throwOnNotFound) {
+                if (logErrors) {
+                    this.logger.warn('No file found at given path', {filePath: path});
+                }
+                throw e;
+            } else {
+                return;
+            }
+        } else if (logErrors) {
+            this.logger.warn(`Encountered error while parsing file`, {filePath: path});
+            this.logger.error(e);
+        }
+        throw e;
+    }
 }
 
 export async function readText(path) {
@@ -49,18 +69,261 @@ export function sleep(ms) {
     return new Promise(resolve => setTimeout(resolve, ms));
 }
 
-export const buildTrackString = (obj) => {
-    const {
-        track: {
-            artists = [],
-            name,
-            id,
-            external_urls: {
-                spotify,
-            } = {}
-        } = {},
-        played_at
-    } = obj;
-    let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(' / ');
-    return `${artistString} - ${name}, played at ${played_at}`
+export const longestString = strings => strings.reduce((acc, curr) => curr.length > acc ? curr.length : acc, 0);
+export const truncateStringArrToLength = (length, truncStr = '...') => {
+    const truncater = truncateStringToLength(length, truncStr);
+    return strings => strings.map(truncater);
+}
+export const truncateStringToLength = (length, truncStr = '...') => str => str.length > length ? `${str.slice(0, length)}${truncStr}` : str;
+
+const defaultTransformer = input => input;
+
+export const buildTrackString = (playObj, options = {}) => {
+    const {
+        include = ['time', 'artist', 'track'],
+        transformers: {
+            artists: artistsFunc = a => a.join(' / '),
+            track: trackFunc = defaultTransformer,
+            time: timeFunc = t => t.local().format(),
+            timeFromNow = t => t.local().fromNow(),
+        } = {}
+    } = options;
+    const {
+        data: {
+            artists,
+            album,
+            track,
+            playDate
+        } = {}
+    } = playObj;
+
+    let str = '';
+    if(include.includes('artist')) {
+        str = `${artistsFunc(artists)}`;
+    }
+    if(include.includes('track')) {
+        if(str !== '') {
+            str = `${str} - ${trackFunc(track)}`;
+        } else {
+            str = `${trackFunc(track)}`;
+        }
+    }
+    if (include.includes('time')) {
+        str = `${str} @ ${timeFunc(playDate)}`
+    }
+    if (include.includes('timeFromNow')) {
+        str = `${str} (${timeFromNow(playDate)})`
+    }
+    return str;
+}
+
+// sorts playObj formatted objects by playDate in ascending (oldest first) order
+export const sortByPlayDate = (a, b) => a.data.playDate.isAfter(b.data.playDate) ? 1 : -1;
+
+const s = splat();
+const SPLAT = Symbol.for('splat')
+const errorsFormat = errors({stack: true});
+const CWD = process.cwd();
+
+let longestLabel = 3;
+export const defaultFormat = printf(({level, message, label = 'App', timestamp, [SPLAT]: splatObj, stack, ...rest}) => {
+    let stringifyValue = splatObj !== undefined ? jsonStringify(splatObj) : '';
+    if (label.length > longestLabel) {
+        longestLabel = label.length;
+    }
+    let msg = message;
+    let stackMsg = '';
+    if (stack !== undefined) {
+        const stackArr = stack.split('\n');
+        msg = stackArr[0];
+        const cleanedStack = stackArr
+            .slice(1) // don't need actual error message since we are showing it as msg
+            .map(x => x.replace(CWD, 'CWD')) // replace file location up to cwd for user privacy
+            .join('\n'); // rejoin with newline to preserve formatting
+        stackMsg = `\n${cleanedStack}`;
+    }
+
+    return `${timestamp} ${level.padEnd(7)}: [${label.padEnd(longestLabel)}] ${msg}${stringifyValue !== '' ? ` ${stringifyValue}` : ''}${stackMsg}`;
+});
+
+export const labelledFormat = (labelName = 'App') => {
+    const l = label({label: labelName, message: false});
+    return combine(
+        timestamp(
+            {
+                format: () => dayjs().local().format(),
+            }
+        ),
+        l,
+        s,
+        errorsFormat,
+        defaultFormat,
+    );
+}
+
+export const createLabelledLogger = (name = 'default', label = 'App') => {
+    if (winston.loggers.has(name)) {
+        return winston.loggers.get(name);
+    }
+    const def = winston.loggers.get('default');
+    winston.loggers.add(name, {
+        transports: def.transports,
+        level: def.level,
+        format: labelledFormat(label)
+    });
+    return winston.loggers.get(name);
+}
+
+export const setIntersection = (setA, setB) => {
+    let _intersection = new Set()
+    for (let elem of setB) {
+        if (setA.has(elem)) {
+            _intersection.add(elem)
+        }
+    }
+    return _intersection
+}
+
+export const isValidConfigStructure = (obj, required = {}) => {
+    const {name = false, type = false, data = true} = required;
+    const errs = [];
+    if (obj.type === undefined && type) {
+        errs.push("'type' must be defined");
+    }
+    if (obj.name === undefined && name) {
+        errs.push("'name' must be defined")
+    }
+    if (obj.data === undefined && data) {
+        errs.push("'data' must be defined");
+    }
+    if (errs.length > 0) {
+        return errs;
+    }
+    return true;
+}
+
+export const returnDuplicateStrings = (arr) => {
+    const alreadySeen = [];
+    const dupes = [];
+
+    arr.forEach(str => alreadySeen[str] ? dupes.push(str) : alreadySeen[str] = true);
+    return dupes;
+}
+
+export const capitalize = (str) => {
+    return str.charAt(0).toUpperCase() + str.slice(1)
+}
+
+export const playObjDataMatch = (a, b) => {
+    const {
+        data: {
+            artists: aArtists = [],
+            album: aAlbum,
+            track: aTrack,
+        } = {},
+        meta: {
+            source: aSource,
+            sourceId: aSourceId,
+        } = {},
+    } = a;
+
+    const {
+        data: {
+            artists: bArtists = [],
+            album: bAlbum,
+            track: bTrack,
+        } = {},
+        meta: {
+            source: bSource,
+            sourceId: bSourceId,
+        } = {},
+    } = b;
+
+    // if sources are the same and both plays have source ids then we can just compare by id
+    if(aSource === bSource && aSourceId !== undefined && bSourceId !== undefined) {
+        if(aSourceId !== bSourceId) {
+            return false;
+        }
+    }
+
+    if (aTrack !== bTrack) {
+        return false;
+    }
+    if (aAlbum !== bAlbum) {
+        return false;
+    }
+    if (aArtists.length !== bArtists.length) {
+        return false;
+    }
+    // check if every artist from either playObj matches (one way or another) with the artists from the other play obj
+    if (!aArtists.every(x => bArtists.includes(x)) && bArtists.every(x => aArtists.includes(x))) {
+        return false
+    }
+
+    return true;
+}
+
+export const parseRetryAfterSecsFromObj = (err) => {
+
+    let raVal;
+
+    if (err instanceof TimeoutError) {
+        return undefined;
+    }
+    if (err instanceof WebapiError || err instanceof Response) {
+        const {headers = {}} = err;
+        raVal = headers['retry-after']
+    }
+    // if (err instanceof Response) {
+    //     const {headers = {}} = err;
+    //     raVal = headers['retry-after']
+    // }
+    const {
+        response: {
+            headers, // returned in superagent error
+        } = {},
+        retryAfter: ra // possible custom property we have set
+    } = err;
+
+    if (ra !== undefined) {
+        raVal = ra;
+    } else if (headers !== null && typeof headers === 'object') {
+        raVal = headers['retry-after'];
+    }
+
+    if (raVal === undefined || raVal === null) {
+        return raVal;
+    }
+
+    // first try to parse as float
+    let retryAfter = Number.parseFloat(raVal);
+    if (!isNaN(retryAfter)) {
+        return retryAfter; // got a number!
+    }
+    // try to parse as date
+    retryAfter = dayjs(retryAfter);
+    if (!dayjs.isDayjs(retryAfter)) {
+        return undefined; // could not parse string if not in ISO 8601 format
+    }
+    // otherwise we got a date! now get the difference the specified retry-after date and now in seconds
+    const diff = retryAfter.diff(dayjs(), 'second');
+
+    if (diff <= 0) {
+        // if diff is in the past returned undefined as its irrelevant now
+        return undefined;
+    }
+
+    return diff;
+}
+
+export const spreadDelay = (retries, multiplier) => {
+    if(retries === 0) {
+        return [];
+    }
+    let r;
+    let s = [];
+    for(r = 0; r < retries; r++) {
+        s.push(((r+1) * multiplier) * 1000);
+    }
+    return s;
 }

views/recent.ejs

@@ -0,0 +1,65 @@
+<html>
+<head>
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.0.3/tailwind.min.css" integrity="sha512-wl80ucxCRpLkfaCnbM88y4AxnutbGk327762eM9E/rRTvY/ZGAHWMZrYUq66VQBYMIYDFpDdJAOGSLyIPHZ2IQ==" crossorigin="anonymous" />
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.0.3/tailwind-dark.min.css" integrity="sha512-WvyKyiVHgInX5UQt67447ExtRRZG/8GUijaq1MpqTNYp8wY4/EJOG5bI80sRp/5crDy4Z6bBUydZI2OFV3Vbtg==" crossorigin="anonymous" />
+    <script type="module" src="https://cdn.jsdelivr.net/npm/@catalyst-elements/catalyst-toggle-switch@0.7.0/catalyst-toggle-switch.min.js"></script>
+    <style>
+        a {
+            text-decoration: underline;
+        }
+    </style>
+    <title>Multi Scrobbler - Status</title>
+</head>
+<body>
+    <div class="min-w-screen min-h-screen bg-gray-100 bg-gray-100 dark:bg-gray-800 font-sans">
+        <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-800 text-white">
+            <div class="container mx-auto">
+                <div class="flex items-center justify-between">
+                    <a href="/" class="flex items-center flex-grow no-underline">
+                        <img src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyODkuNTUgMTM1LjE3Ij48ZGVmcz48c3R5bGU+LmNscy0xe2ZpbGw6IzNkNTNhNDt9LmNscy0ye2ZpbGw6I2ZmZjt9PC9zdHlsZT48L2RlZnM+PGcgaWQ9IkxheWVyXzEiIGRhdGEtbmFtZT0iTGF5ZXIgMSI+PHBhdGggY2xhc3M9ImNscy0xIiBkPSJNMTI5Ljg3LDE3My4zNCwzOS4zOCwyNjMuODNaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0xMTQsMTU3LjQzbC03OS4zOSw3OS40LTExLjEsMTEuMDljLTguMzMsOC4zNC04Ljk1LDIzLjYsMCwzMS44MnMyMi45Myw4LjksMzEuODIsMGw3OS40LTc5LjM5LDExLjA5LTExLjFjOC4zMy04LjMzLDktMjMuNiwwLTMxLjgycy0yMi45My04Ljg5LTMxLjgyLDBsLTc5LjM5LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMDktMTEuMWM4LjMzLTguMzMsOS0yMy42LDAtMzEuODJTMTIyLjg1LDE0OC41NCwxMTQsMTU3LjQzWiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTE2Ljk5IC0xNTEpIi8+PC9nPjxnIGlkPSJMYXllcl8xX2NvcHkiIGRhdGEtbmFtZT0iTGF5ZXIgMSBjb3B5Ij48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0xOTEuODgsMTU3LjQzbC03OS40LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMS0xMS4xYzguMzMtOC4zMyw5LTIzLjYsMC0zMS44MnMtMjIuOTMtOC44OS0zMS44MiwwbC03OS40LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMS0xMS4xYzguMzMtOC4zMyw5LTIzLjYsMC0zMS44MlMyMDAuNzcsMTQ4LjU0LDE5MS44OCwxNTcuNDNaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48L2c+PGcgaWQ9IkxheWVyXzFfY29weV8yIiBkYXRhLW5hbWU9IkxheWVyIDEgY29weSAyIj48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0yNjguMjMsMTU3LjQzbC0yOC43NywyOC43OGMtOC4zNCw4LjMzLTksMjMuNiwwLDMxLjgyczIyLjkyLDguODksMzEuODIsMGwyOC43Ny0yOC43OGM4LjMzLTguMzMsOS0yMy42LDAtMzEuODJzLTIyLjkzLTguODktMzEuODIsMGwtMjguNzcsMjguNzhjLTguMzQsOC4zMy05LDIzLjYsMCwzMS44MnMyMi45Miw4Ljg5LDMxLjgyLDBsMjguNzctMjguNzhjOC4zMy04LjMzLDktMjMuNiwwLTMxLjgyUzI3Ny4xMiwxNDguNTQsMjY4LjIzLDE1Ny40M1oiIHRyYW5zZm9ybT0idHJhbnNsYXRlKC0xNi45OSAtMTUxKSIvPjwvZz48ZyBpZD0iTGF5ZXJfMV9jb3B5XzMiIGRhdGEtbmFtZT0iTGF5ZXIgMSBjb3B5IDMiPjxwYXRoIGNsYXNzPSJjbHMtMiIgZD0iTTIwMy40OSwyMjIuMTcsMTc5LjEsMjQ2LjU2Yy04LjMzLDguMzQtOC45NSwyMy42LDAsMzEuODJzMjIuOTMsOC45LDMxLjgyLDBMMjM1LjMxLDI1NGM4LjM0LTguMzMsOS0yMy42LDAtMzEuODJzLTIyLjkyLTguODktMzEuODIsMEwxNzkuMSwyNDYuNTZjLTguMzMsOC4zNC04Ljk1LDIzLjYsMCwzMS44MnMyMi45Myw4LjksMzEuODIsMEwyMzUuMzEsMjU0YzguMzQtOC4zMyw5LTIzLjYsMC0zMS44MlMyMTIuMzksMjEzLjI4LDIwMy40OSwyMjIuMTdaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48L2c+PC9zdmc+" style="max-width:100px; max-height:30px;"/>
+                        <span class="px-4 break-normal">
+                            Multi Scrobbler
+                        </span>
+                    </a>
+                    <div>
+                        Dark mode <catalyst-toggle-switch id="toggle-switch"></catalyst-toggle-switch>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <div class="container mx-auto">
+            <div class="grid">
+                <div class="bg-white shadow-md rounded my-6 dark:bg-gray-500 dark:text-white">
+                    <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-300 dark:bg-gray-700 dark:text-white">
+                        <h3>Recently Played (<%= name %>)</h3>
+                    </div>
+                    <div class="p-6 md:px-10 md:py-6">
+                        <% if (sourceType === 'spotify') { %>
+                            This list was returned from the Spotify Api <b>now.</b> Sometimes Spotify "backlogs" user activity so your recent tracks may not show up for minutes (or hours). ¯\_(ツ)_/¯
+                        <% } %>
+                        <ul>
+                            <% plays.forEach(function (play){ %>
+                                <li><%-play%></li>
+                            <% }) %>
+                        </ul>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+    <script>
+        let toggleSwitch = document.querySelector('#toggle-switch');
+        toggleSwitch.addEventListener('change', (event) => {
+            document.body.classList.toggle('dark')
+            toggleSwitch.checked ? localStorage.setItem('ms-dark', 'yes') : localStorage.setItem('ms-dark', 'no')
+        });
+        if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
+            if (localStorage.getItem('ms-dark') === 'yes') {
+                document.body.classList.add('dark')
+                toggleSwitch.checked = true
+                localStorage.setItem('ms-dark', 'yes')
+            }
+        }
+    </script>
+</body>
+</html>

views/status.ejs

@@ -0,0 +1,107 @@
+<html>
+<head>
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.0.3/tailwind.min.css" integrity="sha512-wl80ucxCRpLkfaCnbM88y4AxnutbGk327762eM9E/rRTvY/ZGAHWMZrYUq66VQBYMIYDFpDdJAOGSLyIPHZ2IQ==" crossorigin="anonymous" />
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.0.3/tailwind-dark.min.css" integrity="sha512-WvyKyiVHgInX5UQt67447ExtRRZG/8GUijaq1MpqTNYp8wY4/EJOG5bI80sRp/5crDy4Z6bBUydZI2OFV3Vbtg==" crossorigin="anonymous" />
+    <script type="module" src="https://cdn.jsdelivr.net/npm/@catalyst-elements/catalyst-toggle-switch@0.7.0/catalyst-toggle-switch.min.js"></script>
+    <style>
+        a {
+            text-decoration: underline;
+        }
+    </style>
+    <title>Multi Scrobbler</title>
+</head>
+<body class="">
+<script>localStorage.getItem('ms-dark') === 'yes' ? document.body.classList.add('dark') : document.body.classList.remove('dark')</script>
+<div class="min-w-screen min-h-screen bg-gray-100 bg-gray-100 dark:bg-gray-800 font-sans">
+    <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-800 text-white">
+        <div class="container mx-auto">
+            <div class="flex items-center justify-between">
+                <a href="/" class="flex items-center flex-grow no-underline">
+                    <img src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyODkuNTUgMTM1LjE3Ij48ZGVmcz48c3R5bGU+LmNscy0xe2ZpbGw6IzNkNTNhNDt9LmNscy0ye2ZpbGw6I2ZmZjt9PC9zdHlsZT48L2RlZnM+PGcgaWQ9IkxheWVyXzEiIGRhdGEtbmFtZT0iTGF5ZXIgMSI+PHBhdGggY2xhc3M9ImNscy0xIiBkPSJNMTI5Ljg3LDE3My4zNCwzOS4zOCwyNjMuODNaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0xMTQsMTU3LjQzbC03OS4zOSw3OS40LTExLjEsMTEuMDljLTguMzMsOC4zNC04Ljk1LDIzLjYsMCwzMS44MnMyMi45Myw4LjksMzEuODIsMGw3OS40LTc5LjM5LDExLjA5LTExLjFjOC4zMy04LjMzLDktMjMuNiwwLTMxLjgycy0yMi45My04Ljg5LTMxLjgyLDBsLTc5LjM5LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMDktMTEuMWM4LjMzLTguMzMsOS0yMy42LDAtMzEuODJTMTIyLjg1LDE0OC41NCwxMTQsMTU3LjQzWiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTE2Ljk5IC0xNTEpIi8+PC9nPjxnIGlkPSJMYXllcl8xX2NvcHkiIGRhdGEtbmFtZT0iTGF5ZXIgMSBjb3B5Ij48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0xOTEuODgsMTU3LjQzbC03OS40LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMS0xMS4xYzguMzMtOC4zMyw5LTIzLjYsMC0zMS44MnMtMjIuOTMtOC44OS0zMS44MiwwbC03OS40LDc5LjQtMTEuMSwxMS4wOWMtOC4zMyw4LjM0LTguOTUsMjMuNiwwLDMxLjgyczIyLjkzLDguOSwzMS44MiwwbDc5LjQtNzkuMzksMTEuMS0xMS4xYzguMzMtOC4zMyw5LTIzLjYsMC0zMS44MlMyMDAuNzcsMTQ4LjU0LDE5MS44OCwxNTcuNDNaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48L2c+PGcgaWQ9IkxheWVyXzFfY29weV8yIiBkYXRhLW5hbWU9IkxheWVyIDEgY29weSAyIj48cGF0aCBjbGFzcz0iY2xzLTIiIGQ9Ik0yNjguMjMsMTU3LjQzbC0yOC43NywyOC43OGMtOC4zNCw4LjMzLTksMjMuNiwwLDMxLjgyczIyLjkyLDguODksMzEuODIsMGwyOC43Ny0yOC43OGM4LjMzLTguMzMsOS0yMy42LDAtMzEuODJzLTIyLjkzLTguODktMzEuODIsMGwtMjguNzcsMjguNzhjLTguMzQsOC4zMy05LDIzLjYsMCwzMS44MnMyMi45Miw4Ljg5LDMxLjgyLDBsMjguNzctMjguNzhjOC4zMy04LjMzLDktMjMuNiwwLTMxLjgyUzI3Ny4xMiwxNDguNTQsMjY4LjIzLDE1Ny40M1oiIHRyYW5zZm9ybT0idHJhbnNsYXRlKC0xNi45OSAtMTUxKSIvPjwvZz48ZyBpZD0iTGF5ZXJfMV9jb3B5XzMiIGRhdGEtbmFtZT0iTGF5ZXIgMSBjb3B5IDMiPjxwYXRoIGNsYXNzPSJjbHMtMiIgZD0iTTIwMy40OSwyMjIuMTcsMTc5LjEsMjQ2LjU2Yy04LjMzLDguMzQtOC45NSwyMy42LDAsMzEuODJzMjIuOTMsOC45LDMxLjgyLDBMMjM1LjMxLDI1NGM4LjM0LTguMzMsOS0yMy42LDAtMzEuODJzLTIyLjkyLTguODktMzEuODIsMEwxNzkuMSwyNDYuNTZjLTguMzMsOC4zNC04Ljk1LDIzLjYsMCwzMS44MnMyMi45Myw4LjksMzEuODIsMEwyMzUuMzEsMjU0YzguMzQtOC4zMyw5LTIzLjYsMC0zMS44MlMyMTIuMzksMjEzLjI4LDIwMy40OSwyMjIuMTdaIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTYuOTkgLTE1MSkiLz48L2c+PC9zdmc+" style="max-width:100px; max-height:30px;"/>
+                    <span class="px-4 break-normal">
+                        Multi Scrobbler
+                    </span>
+                </a>
+                <div>
+                    Dark mode <catalyst-toggle-switch id="toggle-switch"></catalyst-toggle-switch>
+                </div>
+            </div>
+        </div>
+    </div>
+    <div class="container mx-auto">
+        <div class="grid md:grid-cols-3 gap-3 ">
+            <% sources.forEach(function (source){ %>
+            <div class="bg-white shadow-md rounded my-6 dark:bg-gray-500 dark:text-white">
+                <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-300 dark:bg-gray-700 dark:text-white">
+                    <h3>(Source) <%= source.display %> - <%= source.name %></h3>
+                </div>
+                <div class="p-6 md:px-10 md:py-6">
+                    <div><b>Status: <%= source.status %></b></div>
+                    <div>Tracks Discovered (since app started): <%= source.tracksDiscovered %></div>
+                    <% if (source.canPoll === true) { %>
+                    <div><a target="_blank" <% if (!source.hasAuth || source.authed) { %> href="recent?name=<%= source.name %>&type=<%= source.type %>" <% } else { %>class="disabled"<% } %>>See recently played tracks returned by API</a></div>
+                        <% if (source.hasAuth) { %>
+                    <div><a target="_blank" href="source/auth?name=<%= source.name %>&type=<%= source.type %>">(Re)authenticate and (re)start polling</a></div>
+                        <% } %>
+                    <div><a target="_blank" <% if (!source.hasAuth || source.authed) { %> href="poll?name=<%= source.name %>&type=<%= source.type %>" <% } else { %>class="disabled"<% } %>>Restart polling</a></div>
+                    <% } %>
+                </div>
+            </div>
+            <% }) %>
+        </div>
+        <div class="grid md:grid-cols-3 gap-3">
+            <% clients.forEach(function (client){ %>
+                <div class="bg-white shadow-md rounded my-6 dark:bg-gray-500 dark:text-white">
+                    <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-300 dark:bg-gray-700 dark:text-white">
+                        <h3>(Client) <%= client.display %> - <%= client.name %></h3>
+                    </div>
+                    <div class="p-6 md:px-10 md:py-6">
+                        <ul>
+                            <div><b>Status: <%= client.status %></b></div>
+                            <div>Tracks Scrobbled (since app started): <%= client.tracksDiscovered %></div>
+                            <% if (client.hasAuth) { %>
+                                <div>Click <a target="_blank" href="client/auth?name=<%= client.name %>&type=<%= client.type %>">to (re)authenticate or initialize</a></div>
+                            <% } %>
+                        </ul>
+                    </div>
+                </div>
+            <% }) %>
+        </div>
+        <div class="grid ">
+            <div class="bg-white shadow-md rounded my-6 dark:bg-gray-500 dark:text-white">
+                <div class="space-x-4 p-6 md:px-10 md:py-6 leading-6 font-semibold bg-gray-300 dark:bg-gray-700 dark:text-white">
+                    <h2>Log (Most Recent)</h2>
+                </div>
+                <div class="p-6 md:px-10 md:py-6">
+                    <div>Level : <%- logs.level %> </div>
+                    <div>Sort  : <%- logs.sort %></div>
+                    <div>Limit : <%- logs.limit %></div>
+                    <br />
+                    <% logs.output.forEach(function (logEntry){ %>
+                        <%-logEntry%>
+                    <% }) %>
+                </div>
+                <div class="w-full flex-auto flex min-h-0 overflow-auto">
+                    <div class="w-full relative flex-auto">
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+</div>
+<script>
+    let toggleSwitch = document.querySelector('#toggle-switch');
+    toggleSwitch.addEventListener('change', (event) => {
+        document.body.classList.toggle('dark')
+        toggleSwitch.checked ? localStorage.setItem('ms-dark', 'yes') : localStorage.setItem('ms-dark', 'no')
+    });
+    if (window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches) {
+        if (localStorage.getItem('ms-dark') === 'yes') {
+            document.body.classList.add('dark')
+            toggleSwitch.checked = true
+            localStorage.setItem('ms-dark', 'yes')
+        }
+    }
+</script>
+</body>
+</html>