Fix how scrobbled tracks are returned to source

Returns tracks array should only show that was a track was scrobbled or not (by existing in the array) -- and only be in the array once.

.dockerignore

@@ -5,5 +5,9 @@ Dockerfile
 .dockerignore
 .gitignore
 .git
-spotifyCreds.json
+config/currentCreds.json
 *.log
+config/maloja.json
+config/spotify.json
+config/config.json
+/docs

Dockerfile

@@ -1,5 +1,9 @@
 FROM node:fermium-alpine3.10
 
+ENV TZ=Etc/GMT
+
+RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+
 RUN mkdir -p /home/node/app/node_modules && chown -R node:node /home/node
 
 WORKDIR /home/node/app
@@ -8,7 +12,7 @@ COPY package*.json ./
 
 USER node
 
-RUN npm install
+RUN npm install --production
 
 COPY --chown=node:node . .
 
@@ -19,6 +23,11 @@ RUN mkdir -p $config_dir
 VOLUME $config_dir
 ENV CONFIG_DIR=$config_dir
 
+ARG log_dir=/home/node/logs
+RUN mkdir -p $log_dir
+VOLUME $log_dir
+ENV LOG_DIR=$log_dir
+
 ARG webPort=9078
 ENV PORT=$webPort
 EXPOSE $PORT

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 FoxxMD
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,83 +1,87 @@
-# spotify-scrobbler
+# multi-scrobbler
 
-A single-user, javascript app to scrobble your recent plays to [Maloja](https://github.com/krateng/maloja) (and other clients, eventually)
+[![Latest Release](https://img.shields.io/github/v/release/foxxmd/multi-scrobbler)](https://github.com/FoxxMD/multi-scrobbler/releases)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Docker Pulls](https://img.shields.io/docker/pulls/foxxmd/multi-scrobbler)](https://hub.docker.com/r/foxxmd/multi-scrobbler)
+
+A javascript app to scrobble plays from multiple sources to [Maloja](https://github.com/krateng/maloja), [Last.fm](https://www.last.fm), and other clients (eventually!)
+
+* Supports scrobbling for many sources
+  * [Spotify](/docs/configuration.md#spotify)
+  * [Plex](/docs/configuration.md#plex) or [Tautulli](/docs/configuration.md#tautulli)
+  * [Subsonic-compatible APIs](/docs/configuration.md#subsonic) (like [Airsonic](https://airsonic.github.io/))
+* Supports scrobbling to many clients
+  * [Maloja](/docs/configuration.md#maloja)
+  * [Last.fm](/docs/configuration.md#lastfm)
+* Supports configuring for single or multiple users (scrobbling for your friends and family!)
+* Web server interface for stats, basic control, and detailed logs
+* Smart handling of credentials (persistent, authorization through app)
+* Easy configuration through ENVs or JSON
+* Built for Docker and unattended use!
+
+**Why should I use this over a browser extension and/or mobile app scrobbler?**
+
+* **Platform independent** -- Because multi-scrobbler communicates directly with service APIs it will scrobble everything you play regardless of where you play it. No more need for apps on every platform you use!
+* **Open-source** -- Get peace of mind knowing exactly how your personal data is being handled.
+* **Consolidate play sources** -- Scrobble from many sources to one client with ease and without duplicating tracks.
+* **Manage scrobbling for others** -- Scrobble for your friends and family without any setup on their part. Easily silo sources to specific clients to keep plays separate.
 
 ## Installation
 
+
+### Locally
+
 Clone this repository somewhere and then install from the working directory
 
 ```bash
+git clone https://github.com/FoxxMD/multi-scrobbler.git .
+cd multi-scrobbler
 npm install
 ```
 
-## Setup/Configuration
+### [Docker](https://hub.docker.com/r/foxxmd/multi-scrobbler)
 
-All configuration is done through json files or environment variables. Reference the [examples in the config folder](https://github.com/FoxxMD/spotify-scrobbler/tree/master/config) more detailed explanations and structure.
-
-**A property from a json config will override the environmental variable.**
-
-### General
-
-[JSON config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/config.json.example)
-
-Environment Variables
-* CONFIG_DIR - Default `./config` - Sets configuration directory to look for all other configuration files (if they are not specified)
-* CONFIG_PATH - Default `CONFIG_DIR/config.json`
-* LOG_PATH - Default `true` - If `false` no logs will be written. If `string` will be the directory logs are written to
-* PORT - Default 9078 - Port to run web server on (for authentication callbacks)
-
-
-### Spotify
-
-[Spotify config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/spotify.json.example)
-
-Environment Variables
-* SPOTIFY_CONFIG_PATH - Optional, defaults to `CONFIG_DIR/spotify.json`
-* SPOTIFY_CLIENT_ID - **Required**
-* SPOTIFY_CLIENT_SECRET - **Required**
-* SPOTIFY_ACCESS_TOKEN - Optional if client/secret provided
-* SPOTIFY_REFRESH_TOKEN - Optional if client/secret provided
-* SPOTIFY_REDIRECT_URI - Optional, default is `http://localhost:{port}/callback`
-
-The app will automatically obtain new access/refresh token if needed and possible. These will override values from configuration.
-
-### Maloja
-
-[Maloja config example](https://github.com/FoxxMD/spotify-scrobbler/blob/master/config/maloja.json.example)
-
-Environment Variables
-* MALOJA_CONFIG_PATH - Optional, defaults to `CONFIG_DIR/maloja.json`
-* MALOJA_URL - **Required** - Base Url of your Maloja installation
-* MALOJA_API_KEY - **Required** - Api Key for scrobbling
-
-## Usage
-
-Output is provided to stdout/stderr as well as file if specified in configuration.
-
-On first startup you may need to authroize Spotify by visiting a callback URL. The default url to open is:
-
-```
-https://localhost:9078/authSpotify
-```
-
-### Running Directly
-
-```
-node index.js
-```
-
-### Docker
-
-[Docker repository](https://hub.docker.com/repository/docker/foxxmd/spotify-scrobbler)
 ```
 foxxmd/spotify-scrobbler:latest
 ```
 
-Minimal configuration requires you to bind a host directory for the configuration directory in the container:
+## Setup
+
+Some setup is required! See the [configuration](docs/configuration.md) docs for a full reference.
+
+### TLDR, Minimal Example
+
+You want to use multi-scrobbler to scrobble your plays from Spotify to Maloja:
+
+#### Local
+```bash
+SPOTIFY_CLIENT_ID=yourId SPOTIFY_CLIENT_SECRET=yourSecret MALOJA_URL=http://domain.tld MALOJA_API_KEY=1234 node index.js
+```
+
+#### Docker
+
+```bash
+docker run -e "SPOTIFY_CLIENT_ID=yourId" -e "SPOTIFY_CLIENT_SECRET=yourSecret" -e "MALOJA_URL=http://domain.tld" -e "MALOJA_API_KEY=1234" -v /path/on/host/config:/home/node/app/config foxxmd/spotify-scrobbler
+```
+
+**But I want to use json for configuration?**
+
+Then use [config.json.example](/config/config.json.example) and drop it in your `CONFIG_DIR` directory
+
+**Is there an example configuration using everything?**
+
+Yes, check out the [kitchen sink example](/docs/kitchensink.md)
+
+## Usage
+
+A status page with statistics, recent logs, and some runtime configuration options can be found at
 
 ```
-docker run ... -v /path/on/host/config:/home/node/config ...
+https://localhost:9078
 ```
+Output is also provided to stdout/stderr as well as file if specified in configuration.
+
+On first startup you may need to authorize Spotify by visiting the callback URL (which can also be accessed from the status page). Visit the status page above to find the applicable link to trigger this.
 
 ## License
 

clients/AbstractScrobbleClient.js

@@ -0,0 +1,137 @@
+import dayjs from "dayjs";
+import {buildTrackString, capitalize, createLabelledLogger, playObjDataMatch} from "../utils.js";
+
+export default class AbstractScrobbleClient {
+
+    name;
+    type;
+    initialized = false;
+
+    recentScrobbles = [];
+    scrobbledPlayObjs = [];
+    newestScrobbleTime;
+    oldestScrobbleTime = dayjs();
+    tracksScrobbled = 0;
+
+    lastScrobbleCheck = dayjs();
+    refreshEnabled;
+    checkExistingScrobbles;
+    verboseOptions;
+
+    config;
+    logger;
+
+    constructor(type, name, config = {}) {
+        this.type = type;
+        this.name = name;
+        const identifier = `${capitalize(this.type)} - ${name}`;
+        this.logger = createLabelledLogger(identifier, identifier);
+
+        const {
+            options: {
+                refreshEnabled = true,
+                checkExistingScrobbles = true,
+                verbose = {},
+            } = {},
+            ...rest
+        } = config;
+        this.config = rest;
+        this.refreshEnabled = refreshEnabled;
+        this.checkExistingScrobbles = checkExistingScrobbles;
+
+        const {
+            match: {
+                onNoMatch = false,
+                onMatch = false,
+                confidenceBreakdown = false,
+            } = {},
+            ...vRest
+        } = verbose
+        if (onMatch || onNoMatch) {
+            this.logger.warn('Setting verbose matching may produce noisy logs! Use with care.');
+        }
+        this.verboseOptions = {
+            ...vRest,
+            match: {
+                onNoMatch,
+                onMatch,
+                confidenceBreakdown
+            }
+        };
+    }
+
+    scrobblesLastCheckedAt = () => {
+        return this.lastScrobbleCheck;
+    }
+
+    formatPlayObj = obj => {
+        this.logger.warn('formatPlayObj should be defined by concrete class!');
+        return obj;
+    }
+
+    // time frame is valid as long as the play date for the source track is newer than the oldest play time from the scrobble client
+    // ...this is assuming the scrobble client is returning "most recent" scrobbles
+    timeFrameIsValid = (playObj, log = false) => {
+        const {
+            data: {
+                playDate,
+            } = {},
+        } = playObj;
+        const validTime = playDate.isAfter(this.oldestScrobbleTime);
+        if (log && !validTime) {
+            this.logger.debug(`${buildTrackString(playObj)} was in an invalid time frame (played before the oldest scrobble found)`);
+        }
+        return validTime;
+    }
+
+    addScrobbledTrack = (playObj, scrobbleResp) => {
+        this.scrobbledPlayObjs.push({play: playObj, scrobble: this.formatPlayObj(scrobbleResp)});
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+            } = {},
+        } = playObj;
+
+        return track;
+    };
+
+    findExistingSubmittedPlayObj = (playObj) => {
+        const {
+            data: {
+                playDate
+            } = {},
+            meta: {
+                source,
+            } = {}
+        } = playObj;
+
+        const dtInvariantMatches = this.scrobbledPlayObjs.filter(x => playObjDataMatch(playObj, x.play));
+
+        if (dtInvariantMatches.length === 0) {
+            return [undefined, undefined];
+        }
+
+        const matchPlayDate = dtInvariantMatches.find((x) => {
+            const {
+                play: {
+                    data: {
+                        playDate: sPlayDate
+                    } = {},
+                    meta: {
+                        source: playSource
+                    } = {},
+                } = {},
+            } = x;
+            // need to account for inaccurate DT from subsonic
+            if(source === 'Subsonic' && playSource === 'Subsonic') {
+                return playDate.isSame(sPlayDate) || playDate.diff(sPlayDate, 'minute') <= 1;
+            }
+            return playDate.isSame(sPlayDate);
+        });
+
+        return [matchPlayDate, dtInvariantMatches];
+    }
+}

clients/LastfmScrobbler.js

@@ -0,0 +1,393 @@
+import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
+import dayjs from 'dayjs';
+import LastFm from 'lastfm-node-client';
+import {
+    buildTrackString,
+    playObjDataMatch,
+    readJson,
+    setIntersection, sleep,
+    sortByPlayDate,
+    truncateStringToLength, writeFile
+} from "../utils.js";
+
+const badErrors = [
+    'api key suspended',
+    'invalid session key',
+    'invalid api key',
+    'authentication failed'
+];
+
+const retryErrors = [
+    'operation failed',
+    'service offline',
+    'temporarily unavailable',
+    'rate limit'
+]
+
+export default class LastfmScrobbler extends AbstractScrobbleClient {
+
+    client;
+    redirectUri;
+    workingCredsPath;
+    initialized = false;
+    user;
+
+    constructor(name, config = {}, options = {}) {
+        super('lastfm', name, config, options);
+        const {redirectUri, apiKey, secret, session, configDir} = config;
+        this.redirectUri = `${redirectUri}?state=${name}`;
+        if (apiKey === undefined) {
+            this.logger.warn("'apiKey' not found in config! Client will most likely fail when trying to scrobble");
+        }
+        this.workingCredsPath = `${configDir}/currentCreds-lastfm-${name}.json`;
+        this.client = new LastFm(apiKey, secret, session);
+    }
+
+    static formatPlayObj(obj) {
+        const {
+            artist: {
+                '#text': artists
+            },
+            name: title,
+            album: {
+                '#text': album,
+            },
+            duration,
+            date: {
+                uts: time,
+            },
+        } = obj;
+        let artistStrings = artists.split(',');
+        return {
+            data: {
+                artists: [...new Set(artistStrings)],
+                track: title,
+                album,
+                duration,
+                playDate: dayjs.unix(time),
+            },
+            meta: {
+                source: 'Lastfm',
+            }
+        }
+    }
+
+    formatPlayObj = obj => LastfmScrobbler.formatPlayObj(obj);
+
+    callApi = async (func, tries = 1) => {
+        try {
+            return await func(this.client);
+        } catch (e) {
+            const {
+                message,
+            } = e;
+            // for now check for exceptional errors by matching error code text
+            const retryError = retryErrors.find(x => message.toLocaleLowerCase().includes(x));
+            if(undefined !== retryError) {
+                if(tries <= 2) {
+                    const delay = tries * 3;
+                    this.logger.warn(`API call was not good but recoverable (${retryError}), retrying in ${delay} seconds...`);
+                    await sleep(delay * 1000);
+                    return this.callApi(func, tries + 1);
+                } else {
+                    this.logger.warn('Could not recover!');
+                    throw e;
+                }
+            }
+
+            throw e;
+        }
+    }
+
+    getAuthUrl = () => {
+        const redir = `${this.config.redirectUri}?state=${this.name}`;
+        return `http://www.last.fm/api/auth/?api_key=${this.config.apiKey}&cb=${encodeURIComponent(redir)}`
+    }
+
+    authenticate = async (token) => {
+        const sessionRes = await this.client.authGetSession({token});
+        const {
+            session: {
+                key: sessionKey,
+                name, // username
+            } = {}
+        } = sessionRes;
+        this.client.sessionKey = sessionKey;
+
+        await writeFile(this.workingCredsPath, JSON.stringify({
+            sessionKey,
+        }));
+    }
+
+    initialize = async () => {
+
+        try {
+            const creds = await readJson(this.workingCredsPath, {throwOnNotFound: false});
+            const {sessionKey} = creds || {};
+            if (this.client.sessionKey === undefined && sessionKey !== undefined) {
+                this.client.sessionKey = sessionKey;
+            }
+        } catch (e) {
+            this.logger.warn('Current lastfm credentials file exists but could not be parsed', {path: this.workingCredsPath});
+        }
+
+        if (this.client.sessionKey === undefined) {
+            this.logger.info('No session key found. User interaction for authentication required.');
+            return;
+        }
+        try {
+            const infoResp = await this.callApi(client => client.userGetInfo());
+            const {
+                user: {
+                    name,
+                } = {}
+            } = infoResp;
+            this.user = name;
+            this.initialized = true;
+            this.logger.info(`Client authorized for user ${name}`)
+            return true;
+        } catch (e) {
+            this.logger.error('Testing connection failed');
+            return false;
+        }
+    }
+
+    refreshScrobbles = async () => {
+        if (this.refreshEnabled) {
+            this.logger.debug('Refreshing recent scrobbles');
+            const resp = await this.callApi(client => client.userGetRecentTracks({user: this.user, limit: 20}));
+            const {
+                recenttracks: {
+                    track: list = [],
+                }
+            } = resp;
+            this.recentScrobbles = list.map(x => LastfmScrobbler.formatPlayObj(x)).sort(sortByPlayDate);
+            if (this.recentScrobbles.length > 0) {
+                const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
+                const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
+                this.newestScrobbleTime = newestScrobbleTime;
+                this.oldestScrobbleTime = oldestScrobbleTime;
+
+                this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
+            }
+        }
+        this.lastScrobbleCheck = dayjs();
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+            } = {},
+        } = playObj;
+        return track.toLocaleLowerCase().trim();
+    }
+
+    alreadyScrobbled = (playObj, log = false) => {
+        return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
+    }
+
+    existingScrobble = (playObj, logMatch = false) => {
+        const tr = truncateStringToLength(27);
+        const scoreTrackOpts = {include: ['track', 'time'], transformers: {track: t => tr(t).padEnd(30)}};
+
+        // return early if we don't care about checking existing
+        if (false === this.checkExistingScrobbles) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
+            }
+            return undefined;
+        }
+
+        let existingScrobble;
+        let closestMatch = {score: 0, breakdowns: ['None']};
+
+        // then check if we have already recorded this
+        const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
+
+        // if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
+        if (existingExactSubmitted !== undefined) {
+            existingScrobble = existingExactSubmitted.scrobble;
+
+            closestMatch = {
+                score: 1,
+                breakdowns: ['Exact Match found in previously successfully scrobbled']
+            }
+        }
+        // if not though then we need to check recent scrobbles from scrobble api.
+        // this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
+
+        // if no recent scrobbles found then assume we haven't submitted it
+        // (either user doesnt want to check history or there is no history to check!)
+        if (this.recentScrobbles.length === 0) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
+            }
+            return undefined;
+        }
+
+        if (existingScrobble === undefined) {
+
+            // we have have found an existing submission but without an exact date
+            // in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
+            const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
+
+            const {
+                data: {
+                    artists: sourceArtists = [],
+                    playDate
+                } = {},
+                meta: {
+                    trackLength,
+                    source,
+                } = {},
+            } = playObj;
+
+            // clean source title so it matches title from the scrobble api response as closely as we can get it
+            let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
+
+            existingScrobble = this.recentScrobbles.find((x) => {
+
+                const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
+
+                const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
+
+                const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
+                let closeTime = false;
+                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
+                let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
+                let scrobblePlayStartDiff;
+                if (scrobblePlayDiff <= playDiffThreshold) {
+                    //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
+                    closeTime = true;
+                }
+                // also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
+                if (closeTime === false && trackLength !== undefined) {
+                    scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
+                    if (scrobblePlayStartDiff <= playDiffThreshold) {
+                        //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
+                        closeTime = true;
+                    }
+                }
+
+                let titleMatch;
+                const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
+                // because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
+                // and compare against length of scrobble title
+                const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter(x => x !== ''));
+                const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
+
+                titleMatch = commonTerms.size / sourceTitleTerms.size;
+
+                let artistMatch;
+                const lowerSourceArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+                const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
+                artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
+
+                const artistScore = .2 * artistMatch;
+                const titleScore = .3 * titleMatch;
+                const timeScore = .5 * (closeTime ? 1 : 0);
+                const referenceScore = .5 * (referenceMatch ? 1 : 0);
+                const score = artistScore + titleScore + timeScore;
+
+                let scoreBreakdowns = [
+                    `Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
+                    `Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
+                    `Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
+                    `Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
+                    `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+                ];
+
+                const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+
+                const scoreInfo = {
+                    score,
+                    scrobble: x,
+                    breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
+                }
+
+                if (closestMatch.score <= score && score > 0) {
+                    closestMatch = scoreInfo
+                }
+
+                return score >= .7;
+            });
+        }
+
+        if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
+            const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
+            this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
+        }
+        return existingScrobble;
+    }
+
+    scrobble = async (playObj) => {
+        const {
+            data: {
+                artists,
+                album,
+                track,
+                duration,
+                playDate
+            } = {},
+            data = {},
+            meta: {
+                source,
+                newFromSource = false,
+            } = {}
+        } = playObj;
+
+        const sType = newFromSource ? 'New' : 'Backlog';
+
+        try {
+            const response = await this.callApi(client => client.trackScrobble(
+                {
+                    artist: artists.join(', '),
+                    duration,
+                    track,
+                    album,
+                    timestamp: playDate.unix(),
+                }));
+            const {
+                scrobbles: {
+                    '@attr': {
+                        accepted = 0,
+                        ignored = 0,
+                        code,
+                    },
+                    scrobble: {
+                        track: {
+                           '#text': trackName,
+                        } = {},
+                        timestamp,
+                        ignoredMessage: {
+                            code: ignoreCode,
+                            '#text': ignoreMsg,
+                        } = {},
+                        ...rest
+                    } = {}
+                } = {},
+            } = response;
+            if(code === 5) {
+                this.initialized = false;
+                throw new Error('Service reported daily scrobble limit exceeded! 😬 Disabling client');
+            }
+            this.addScrobbledTrack(playObj, {...rest, date: { uts: timestamp}, name: trackName});
+            if (newFromSource) {
+                this.logger.info(`Scrobbled (New)     => (${source}) ${buildTrackString(playObj)}`);
+            } else {
+                this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
+            }
+            if(ignoreMsg !== '') {
+                this.logger.warn(`Service ignored this scrobble 😬 => (Code ${ignoreCode}) ${ignoreMsg}`)
+            }
+            // last fm has rate limits but i can't find a specific example of what that limit is. going to default to 1 scrobble/sec to be safe
+            await sleep(1000);
+        } catch (e) {
+            this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj)});
+            throw e;
+        }
+
+        return true;
+    }
+}

clients/MalojaScrobbler.js

@@ -1,74 +1,359 @@
-import ScrobbleClient from "./ScrobbleClient.js";
+import AbstractScrobbleClient from "./AbstractScrobbleClient.js";
 import request from 'superagent';
-import format from 'date-fns/format/index.js';
+import dayjs from 'dayjs';
+import {buildTrackString, playObjDataMatch, setIntersection, sortByPlayDate, truncateStringToLength} from "../utils.js";
 
-export default class MalojaScrobbler extends ScrobbleClient {
+const feat = ["ft.", "ft", "feat.", "feat", "featuring", "Ft.", "Ft", "Feat.", "Feat", "Featuring"];
 
-    name = 'Maloja';
+export default class MalojaScrobbler extends AbstractScrobbleClient {
 
-    refreshScrobbles = async () => {
-        const {url} = this.config;
-        const resp = await request.get(`${url}/apis/mlj_1/scrobbles?since=${format(new Date(), 'yyyy/MM/dd')}&to=${format(new Date(), 'yyyy/MM/dd')}`)
-        this.recentScrobbles = resp.body.list.slice(0, 10);
-        this.lastScrobbleCheck = new Date();
+    constructor(name, config = {}, options = {}) {
+        super('maloja', name, config, options);
+        const {url, apiKey} = config;
+        if (apiKey === undefined) {
+            this.logger.warn("'apiKey' not found in config! Client will most likely fail when trying to scrobble");
+        }
+        if (url === undefined) {
+            throw new Error("Missing 'url' for Maloja config");
+        }
     }
 
-    alreadyScrobbled = (title, playDate, duration) => {
-        const playUnix = playDate.getTime() / 1000;
-        const lowerTitle = title.toLocaleLowerCase();
-        return this.recentScrobbles.some((x) => {
-            const {time: scrobbleTime, title: scrobbleTitle} = x;
-            const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase();
-            if (lowerTitle.includes(lowerScrobbleTitle) || lowerScrobbleTitle.includes(lowerTitle)) {
-                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
-                let scrobblePlayDiff = Math.abs(playUnix - scrobbleTime);
-                if (scrobblePlayDiff < 10) {
-                    this.logger.debug(`Scrobble with same name found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`, {label: this.name});
-                    return true;
-                }
-                // also need to check that scrobble time isn't the BEGINNING of the track
-                let scrobblePlayStartDiff = Math.abs(playUnix - (scrobbleTime - duration));
-                if (scrobblePlayStartDiff < 10) {
-                    this.logger.debug(`Scrobble with same name found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`, {label: this.name});
-                    return true;
-                }
-                this.logger.debug(`Scrobble with same name found but the start/finish times vs scrobble time diffs were too large to consider dups (Start Diff ${scrobblePlayStartDiff}s) (End Diff ${scrobblePlayDiff}s)`, {label: this.name});
+    static formatPlayObj(obj) {
+        const {
+            artists,
+            title,
+            album,
+            duration,
+            time,
+        } = obj;
+        let artistStrings = artists.reduce((acc, curr) => {
+            let aString;
+            if (typeof curr === 'string') {
+                aString = curr;
+            } else if (typeof curr === 'object') {
+                aString = curr.name;
+            }
+            const aStrings = aString.split(',');
+            return [...acc, ...aStrings];
+        }, []);
+        return {
+            data: {
+                artists: [...new Set(artistStrings)],
+                track: title,
+                album,
+                duration,
+                playDate: dayjs.unix(time),
+            },
+            meta: {
+                source: 'Maloja',
+            }
+        }
+    }
+
+    formatPlayObj = obj => MalojaScrobbler.formatPlayObj(obj);
+
+    callApi = async (req) => {
+        try {
+            return await req;
+        } catch (e) {
+            const {
+                message,
+                response: {
+                    status,
+                    body,
+                    text,
+                } = {},
+                response,
+            } = e;
+            let msg = response !== undefined ? `API Call failed: Server Response => ${message}` : `API Call failed: ${message}`;
+            const responseMeta = body ?? text;
+            this.logger.error(msg, {status, response: responseMeta});
+            throw e;
+        }
+    }
+
+    testConnection = async () => {
+
+        const {url, apiKey} = this.config;
+        try {
+            const serverInfoResp = await this.callApi(request.get(`${url}/apis/mlj_1/serverinfo`));
+            const {
+                body: {
+                    version = [],
+                    versionstring = '',
+                } = {},
+            } = serverInfoResp;
+            if (version.length === 0) {
+                this.logger.error('Server did not respond with a version. Either the base URL is incorrect or this Maloja server is too old :(');
                 return false;
             }
+            this.logger.info(`Maloja Server Version: ${versionstring}`);
+            if (version[0] < 2 || version[1] < 7) {
+                this.logger.warn('Maloja Server Version is less than 2.7, please upgrade to ensure compatibility');
+            }
+
+            const resp = await this.callApi(request
+                .get(`${url}/apis/mlj_1/test`)
+                .query({key: apiKey}));
+
+            const {
+                status,
+                body: {
+                    status: bodyStatus,
+                } = {},
+                body = {},
+                text = '',
+            } = resp;
+            if (bodyStatus.toLocaleLowerCase() === 'ok') {
+                this.logger.info('Test connection succeeded!');
+                this.initialized = true;
+                return true;
+            }
+            this.logger.error('Testing connection failed => Server Response body was malformed -- should have returned "status: ok"...is the URL correct?', {
+                status,
+                body,
+                text: text.slice(0, 50)
+            });
             return false;
-        })
+        } catch (e) {
+            this.logger.error('Testing connection failed');
+            return false;
+        }
+    }
+
+    refreshScrobbles = async () => {
+        if (this.refreshEnabled) {
+            this.logger.debug('Refreshing recent scrobbles');
+            const {url} = this.config;
+            const resp = await this.callApi(request.get(`${url}/apis/mlj_1/scrobbles?max=20`));
+            const {
+                body: {
+                    list = [],
+                } = {},
+            } = resp;
+            this.recentScrobbles = list.map(x => MalojaScrobbler.formatPlayObj(x)).sort(sortByPlayDate);
+            if (this.recentScrobbles.length > 0) {
+                const [{data: {playDate: newestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(-1);
+                const [{data: {playDate: oldestScrobbleTime = dayjs()} = {}} = {}] = this.recentScrobbles.slice(0, 1);
+                this.newestScrobbleTime = newestScrobbleTime;
+                this.oldestScrobbleTime = oldestScrobbleTime;
+
+                this.scrobbledPlayObjs = this.scrobbledPlayObjs.filter(x => this.timeFrameIsValid(x.play));
+            }
+        }
+        this.lastScrobbleCheck = dayjs();
+    }
+
+    cleanSourceSearchTitle = (playObj) => {
+        const {
+            data: {
+                track,
+                artists: sourceArtists = [],
+            } = {},
+        } = playObj;
+        let lowerTitle = track.toLocaleLowerCase();
+        lowerTitle = feat.reduce((acc, curr) => acc.replace(curr, ''), lowerTitle);
+        // also remove [artist] from the track if found since that gets removed as well
+        const lowerArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+        lowerTitle = lowerArtists.reduce((acc, curr) => acc.replace(curr, ''), lowerTitle);
+
+        // remove any whitespace in parenthesis
+        lowerTitle = lowerTitle.replace("\\s+(?=[^()]*\\))", '')
+            // replace parenthesis
+            .replace('()', '')
+            .replace('( )', '')
+            .trim();
+
+        return lowerTitle;
+    }
+
+    alreadyScrobbled = (playObj, log = false) => {
+        return this.existingScrobble(playObj, (log || this.verboseOptions.match.onMatch)) !== undefined;
+    }
+
+    existingScrobble = (playObj, logMatch = false) => {
+        const tr = truncateStringToLength(27);
+        const scoreTrackOpts = {include: ['track', 'time'], transformers: {track: t => tr(t).padEnd(30)}};
+
+        // return early if we don't care about checking existing
+        if (false === this.checkExistingScrobbles) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => No Match because existing scrobble check is FALSE`);
+            }
+            return undefined;
+        }
+
+        let existingScrobble;
+        let closestMatch = {score: 0, breakdowns: ['None']};
+
+        // then check if we have already recorded this
+        const [existingExactSubmitted, existingDataSubmitted = []] = this.findExistingSubmittedPlayObj(playObj);
+
+        // if we have an submitted play with matching data and play date then we can just return the response from the original scrobble
+        if (existingExactSubmitted !== undefined) {
+            existingScrobble = existingExactSubmitted.scrobble;
+
+            closestMatch = {
+                score: 1,
+                breakdowns: ['Exact Match found in previously successfully scrobbled']
+            }
+        }
+        // if not though then we need to check recent scrobbles from scrobble api.
+        // this will be less accurate than checking existing submitted (obv) but will happen if backlogging or on a fresh server start
+
+        // if no recent scrobbles found then assume we haven't submitted it
+        // (either user doesnt want to check history or there is no history to check!)
+        if (this.recentScrobbles.length === 0) {
+            if (this.verboseOptions.match.onNoMatch) {
+                this.logger.debug(`(Existing Check) ${buildTrackString(playObj, scoreTrackOpts)} => No Match because no recent scrobbles returned from API`);
+            }
+            return undefined;
+        }
+
+        if (existingScrobble === undefined) {
+
+            // we have have found an existing submission but without an exact date
+            // in which case we can check the scrobble api response against recent scrobbles (also from api) for a more accurate comparison
+            const referenceApiScrobbleResponse = existingDataSubmitted.length > 0 ? existingDataSubmitted[0].scrobble : undefined;
+
+            const {
+                data: {
+                    artists: sourceArtists = [],
+                    playDate
+                } = {},
+                meta: {
+                    trackLength,
+                    source,
+                } = {},
+            } = playObj;
+
+            // clean source title so it matches title from the scrobble api response as closely as we can get it
+            let cleanSourceTitle = this.cleanSourceSearchTitle(playObj);
+
+            existingScrobble = this.recentScrobbles.find((x) => {
+
+                const referenceMatch = referenceApiScrobbleResponse !== undefined && playObjDataMatch(x, referenceApiScrobbleResponse);
+
+                const {data: {playDate: scrobbleTime, track: scrobbleTitle, artists = []} = {}} = x;
+
+                const playDiffThreshold = source === 'Subsonic' ? 60 : 10;
+                let closeTime = false;
+                // check if scrobble time is same as play date (when the track finished playing AKA entered recent tracks)
+                let scrobblePlayDiff = Math.abs(playDate.unix() - scrobbleTime.unix());
+                let scrobblePlayStartDiff;
+                if (scrobblePlayDiff <= playDiffThreshold) {
+                    //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (finish time) vs. scrobble time diff was smaller than 10 seconds`);
+                    closeTime = true;
+                }
+                // also need to check that scrobble time isn't the BEGINNING of the track -- if the source supports durations
+                if (closeTime === false && trackLength !== undefined) {
+                    scrobblePlayStartDiff = Math.abs(playDate.unix() - (scrobbleTime.unix() - trackLength));
+                    if (scrobblePlayStartDiff <= playDiffThreshold) {
+                        //this.logger.debug(`Scrobble with same name (${scrobbleTitle}) found and the play (start time) vs. scrobble time diff was smaller than 10 seconds`);
+                        closeTime = true;
+                    }
+                }
+
+                let titleMatch;
+                const lowerScrobbleTitle = scrobbleTitle.toLocaleLowerCase().trim();
+                // because of all this replacing we need a more position-agnostic way of comparing titles so use intersection on title split by spaces
+                // and compare against length of scrobble title
+                const sourceTitleTerms = new Set(cleanSourceTitle.split(' ').filter(x => x !== ''));
+                const commonTerms = setIntersection(new Set(lowerScrobbleTitle.split(' ')), sourceTitleTerms);
+
+                titleMatch = commonTerms.size / sourceTitleTerms.size;
+
+                let artistMatch;
+                const lowerSourceArtists = sourceArtists.map(x => x.toLocaleLowerCase());
+                const lowerScrobbleArtists = artists.map(x => x.toLocaleLowerCase());
+                artistMatch = setIntersection(new Set(lowerScrobbleArtists), new Set(lowerSourceArtists)).size / artists.length;
+
+                const artistScore = .2 * artistMatch;
+                const titleScore = .3 * titleMatch;
+                const timeScore = .5 * (closeTime ? 1 : 0);
+                const referenceScore = .5 * (referenceMatch ? 1 : 0);
+                const score = artistScore + titleScore + timeScore;
+
+                let scoreBreakdowns = [
+                    `Reference: ${(referenceMatch ? 1 : 0)} * .5 = ${referenceScore.toFixed(2)}`,
+                    `Artist ${artistMatch.toFixed(2)} * .2 = ${artistScore.toFixed(2)}`,
+                    `Title: ${titleMatch.toFixed(2)} * .3 = ${titleScore.toFixed(2)}`,
+                    `Time: ${closeTime ? 1 : 0} * .5 = ${timeScore.toFixed(2)}`,
+                    `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+                ];
+
+                const confidence = `Score ${score.toFixed(2)} => ${score >= .7 ? 'Matched!' : 'No Match'}`
+
+                const scoreInfo = {
+                    score,
+                    scrobble: x,
+                    breakdowns: this.verboseOptions.match.confidenceBreakdown ? scoreBreakdowns : [confidence]
+                }
+
+                if (closestMatch.score <= score && score > 0) {
+                    closestMatch = scoreInfo
+                }
+
+                return score >= .7;
+            });
+        }
+
+        if ((existingScrobble !== undefined && this.verboseOptions.match.onMatch) || (existingScrobble === undefined && this.verboseOptions.match.onNoMatch)) {
+            const closestScrobble = closestMatch.scrobble === undefined ? closestMatch.breakdowns.join(' | ') : `Closest Scrobble: ${buildTrackString(closestMatch.scrobble, scoreTrackOpts)} => ${closestMatch.breakdowns.join(' | ')}`;
+            this.logger.debug(`(Existing Check) Source: ${buildTrackString(playObj, scoreTrackOpts)} => ${closestScrobble}`);
+        }
+        return existingScrobble;
     }
 
     scrobble = async (playObj) => {
         const {url, apiKey} = this.config;
 
         const {
-            track: {
-                artists = [],
-                name,
-                id,
-                external_urls: {
-                    spotify,
-                } = {}
+            data: {
+                artists,
+                album,
+                track,
+                duration,
+                playDate
             } = {},
-            played_at
+            meta: {
+                source,
+                newFromSource = false,
+            } = {}
         } = playObj;
 
-        let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(',');
-        const time = new Date(played_at);
+        const sType = newFromSource ? 'New' : 'Backlog';
+
         try {
-            await request.post(`${url}/apis/mlj_1/newscrobble`)
+            const response = await this.callApi(request.post(`${url}/apis/mlj_1/newscrobble`)
                 .type('json')
                 .send({
-                    artist: artistString,
-                    title: name,
+                    // maloja seems to detect this deliminator much better than commas
+                    // also less likely artist has a forward slash in their name than a comma
+                    artist: artists.join(' / '),
+                    seconds: duration,
+                    title: track,
+                    album,
                     key: apiKey,
-                    time: time.getTime() / 1000
-                });
-            this.logger.info('Scrobbled', {label: this.name});
+                    time: playDate.unix(),
+                }));
+            const {body: {
+                track: {
+                    time: mTime = playDate.unix(),
+                    duration: mDuration = duration,
+                    album: mAlbum = album,
+                    ...rest
+                }
+            } = {}} = response;
+            this.addScrobbledTrack(playObj, {...rest, album: mAlbum, time: mTime, duration: mDuration});
+            if (newFromSource) {
+                this.logger.info(`Scrobbled (New)     => (${source}) ${buildTrackString(playObj)}`);
+            } else {
+                this.logger.info(`Scrobbled (Backlog) => (${source}) ${buildTrackString(playObj)}`);
+            }
         } catch (e) {
-            this.logger.error('Error while scrobbling', {label: this.name});
-            this.logger.log(e);
+            this.logger.error(`Scrobble Error (${sType})`, {playInfo: buildTrackString(playObj)});
+            throw e;
         }
 
         return true;

clients/ScrobbleClient.js

@@ -1,20 +0,0 @@
-export default class ScrobbleClient {
-
-    name;
-
-    recentScrobbles = [];
-
-    lastScrobbleCheck = new Date();
-
-    config;
-    logger;
-
-    constructor(logger, config = {}) {
-        this.logger = logger;
-        this.config = config;
-    }
-
-    scrobblesLastCheckedAt = () => {
-        return this.lastScrobbleCheck;
-    }
-}

clients/ScrobbleClients.js

@@ -0,0 +1,269 @@
+import dayjs from "dayjs";
+import {
+    createLabelledLogger,
+    isValidConfigStructure,
+    playObjDataMatch,
+    readJson,
+    returnDuplicateStrings
+} from "../utils.js";
+import MalojaScrobbler from "./MalojaScrobbler.js";
+import LastfmScrobbler from "./LastfmScrobbler.js";
+
+export default class ScrobbleClients {
+
+    clients = [];
+    logger;
+    configDir;
+
+    clientTypes = ['maloja','lastfm'];
+
+    constructor(configDir) {
+        this.configDir = configDir;
+        this.logger = createLabelledLogger('scrobblers', 'Scrobblers');
+    }
+
+    getByName = (name) => {
+        return this.clients.find(x => x.name === name);
+    }
+
+    buildClientsFromConfig = async () => {
+        let configs = [];
+
+        let configFile;
+        try {
+            configFile = await readJson(`${this.configDir}/config.json`, {throwOnNotFound: false});
+        } catch (e) {
+            throw new Error('config.json could not be parsed');
+        }
+        if (configFile !== undefined) {
+            const {clients: mainConfigClientConfigs = []} = configFile;
+            if (!mainConfigClientConfigs.every(x => x !== null && typeof x === 'object')) {
+                throw new Error('All clients from config.json must be objects');
+            }
+            for (const c of mainConfigClientConfigs) {
+                const {name = 'unnamed'} = c;
+                configs.push({...c, name, source: 'config.json'});
+            }
+        }
+
+        for (const clientType of this.clientTypes) {
+            switch (clientType) {
+                case 'maloja':
+                    // env builder for single user mode
+                    const url = process.env.MALOJA_URL;
+                    const apiKey = process.env.MALOJA_API_KEY;
+                    if (url !== undefined || apiKey !== undefined) {
+                        configs.push({
+                            type: 'maloja',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: {
+                                url,
+                                apiKey
+                            }
+                        })
+                    }
+                    break;
+                case 'lastfm':
+                    const lfm = {
+                        apiKey: process.env.LASTFM_API_KEY,
+                        secret: process.env.LASTFM_SECRET,
+                        redirectUri: process.env.LASTFM_REDIRECT_URI,
+                        session: process.env.LASTFM_SESSION,
+                    };
+                    if (!Object.values(lfm).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'lastfm',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: lfm
+                        })
+                    }
+                    break;
+                default:
+                    break;
+            }
+            let rawClientConfigs;
+            try {
+                rawClientConfigs = await readJson(`${this.configDir}/${clientType}.json`, {throwOnNotFound: false});
+            } catch (e) {
+                throw new Error(`${clientType}.json config file could not be parsed`);
+            }
+            if (rawClientConfigs !== undefined) {
+                let clientConfigs = [];
+                if (Array.isArray(rawClientConfigs)) {
+                    clientConfigs = rawClientConfigs;
+                } else if (rawClientConfigs === null || typeof rawClientConfigs === 'object') {
+                    // backwards compatibility, assuming its single-user mode
+                    if (rawClientConfigs.data === undefined) {
+                        clientConfigs = [{data: rawClientConfigs, mode: 'single', name: 'unnamed'}];
+                    } else {
+                        clientConfigs = [rawClientConfigs];
+                    }
+                } else {
+                    throw new Error(`All top level data from ${clientType}.json must be an object or array of objects`);
+                }
+                for (const m of clientConfigs) {
+                    if (m === null || typeof m !== 'object') {
+                        throw new Error(`All top-level data from ${clientType}.json must be an object or array of objects`);
+                    }
+                    m.source = `${clientType}.json`;
+                    m.type = clientType;
+                    configs.push(m);
+                }
+            }
+        }
+
+        // we have all possible client configurations so we'll check they are minimally valid
+        const configErrors = configs.reduce((acc, c) => {
+            const isValid = isValidConfigStructure(c, {type: true, data: true});
+            if (isValid !== true) {
+                const msg = `Client config from ${c.source} with name [${c.name || 'unnamed'}] of type [${c.type || 'unknown'}] has errors: ${isValid.join(' | ')}`;
+                return acc.concat(msg);
+            }
+            return acc;
+        }, []);
+        if (configErrors.length > 0) {
+            for (const m of configErrors) {
+                this.logger.error(m);
+            }
+            throw new Error('Could not build clients due to above errors');
+        }
+
+        // all client configs are minimally valid
+        // now check that names are unique
+        const nameGroupedConfigs = configs.reduce((acc, curr) => {
+            const {name = 'unnamed'} = curr;
+            const {[name]: n = []} = acc;
+            return {...acc, [name]: [...n, curr]};
+        }, {});
+        let nameErrors = false;
+        for (const [name, configs] of Object.entries(nameGroupedConfigs)) {
+            if (configs.length > 1) {
+                const sources = configs.map(c => `Config object from ${c.source} of type [${c.type}]`);
+                this.logger.error(`Client config naming conflicts -- the following configs have the same name "${name}": 
+${sources.join('\n')}`);
+                nameErrors = true;
+                if (name === 'unnamed') {
+                    this.logger.info('HINT: "unnamed" configs occur when using ENVs, if a multi-user mode config does not have a "name" property, or if a config is built in single-user mode');
+                }
+            }
+        }
+        if (nameErrors) {
+            throw new Error('Could not build clients due to naming conflicts');
+        }
+
+        // finally! all configs are valid, structurally, and can now be passed to addClient
+        // just need to re-map unnnamed to default
+        const finalConfigs = configs.map(({name = 'unnamed', ...x}) => ({
+            ...x,
+            name
+        }));
+        for (const c of finalConfigs) {
+            await this.addClient(c);
+        }
+    }
+
+    addClient = async (clientConfig) => {
+        const isValidConfig = isValidConfigStructure(clientConfig, {name: true, data: true, type: true});
+        if (isValidConfig !== true) {
+            throw new Error(`Config object from ${clientConfig.source || 'unknown'} with name [${clientConfig.name || 'unnamed'}] of type [${clientConfig.type || 'unknown'}] has errors: ${isValidConfig.join(' | ')}`)
+        }
+        const {type, name, data = {}} = clientConfig;
+        switch (type) {
+            case 'maloja':
+                this.logger.debug(`(${name}) Attempting Maloja initialization...`);
+                const mj = new MalojaScrobbler(name, data);
+                const testSuccess = await mj.testConnection();
+                if (testSuccess === false) {
+                    throw new Error(`(${name}) Maloja client not initialized due to failure during connection testing`);
+                } else {
+                    this.logger.info(`(${name}) Maloja client initialized`);
+                    this.clients.push(mj)
+                }
+                break;
+            case 'lastfm':
+                this.logger.debug(`(${name}) Attempting Lastfm initialization...`);
+                const lfm = new LastfmScrobbler(name, {...data, configDir: this.configDir});
+                try {
+                    await lfm.initialize()
+                    this.logger.info(`(${name}) Lastfm client initialized`);
+                    this.clients.push(lfm)
+                } catch(e) {
+                    this.logger.info(`(${name}) Could not initialize Lastfm client`)
+                }
+                break;
+            default:
+                break;
+        }
+    }
+
+    /**
+     * @param {*} data
+     * @param {{scrobbleFrom, scrobbleTo, forceRefresh: boolean}|{scrobbleFrom, scrobbleTo}} options
+     * @returns {Array}
+     */
+    scrobble = async (data, options = {}) => {
+        const playObjs = Array.isArray(data) ? data : [data];
+        const {
+            forceRefresh = false,
+            checkTime = dayjs(),
+            scrobbleTo = [],
+            scrobbleFrom = 'source',
+        } = options;
+
+        const tracksScrobbled = [];
+
+        if (this.clients.length === 0) {
+            this.logger.warn('Cannot scrobble! No clients are configured.');
+        }
+
+        for (const client of this.clients) {
+            if (scrobbleTo.length > 0 && !scrobbleTo.includes(client.name)) {
+                this.logger.debug(`Client '${client.name}' was filtered out by '${scrobbleFrom}'`);
+                continue;
+            }
+            if(client.initialized === false) {
+                this.logger.debug(`Client '${client.name}' is not yet initialized (check authorization?)`);
+                continue;
+            }
+
+                if (forceRefresh || client.scrobblesLastCheckedAt().unix() < checkTime.unix()) {
+                    try {
+                        await client.refreshScrobbles();
+                    } catch(e) {
+                        this.logger.error(`Encountered error while refreshing scrobbles for ${client.name}`);
+                        this.logger.error(e);
+                    }
+                }
+                for (const playObj of playObjs) {
+                    try {
+                        const {
+                            meta: {
+                                newFromSource = false,
+                            } = {}
+                        } = playObj;
+                        if (client.timeFrameIsValid(playObj, newFromSource) && !client.alreadyScrobbled(playObj, newFromSource)) {
+                            await client.scrobble(playObj)
+                            client.tracksScrobbled++;
+                            // since this is what we return to the source only add to tracksScrobbled if not already in array
+                            // (source should only know that a track was scrobbled (binary) -- doesn't care if it was scrobbled more than once
+                            if(!tracksScrobbled.some(x => playObjDataMatch(x, playObj) && x.data.playDate === playObj.data.playDate)) {
+                                tracksScrobbled.push(playObj);
+                            }
+                        }
+                    } catch(e) {
+                        this.logger.error(`Encountered error while in scrobble loop for ${client.name}`);
+                        this.logger.error(e);
+                        // for now just stop scrobbling plays for this client and move on. the client should deal with logging the issue
+                        if(e.continueScrobbling !== true) {
+                            break;
+                        }
+                    }
+                }
+        }
+        return tracksScrobbled;
+    }
+}

config/config.json.example

@@ -1,23 +1,23 @@
 {
-  // string specifies a directory location to log to.
-  // boolean FALSE means do not log to file
-  // boolean TRUE means log to working directory
-  "logPath": true, // optional
-  "interval": 60, // optional, number of seconds to wait before checking spotify for new tracks
-  "port": 9078, // optional, port for server to start on
-  "spotify": 'object or string', // can specify full spotify config object here OR a string location of json file OR do not include to use config dir location
-  // array of client configurations
+  "sources": [
+    {
+      "type": "spotify",          // required, source type
+      "clients": ["myConfig"],    // optional, a list of Client config names this Source should scrobble to. Using an empty list or not including this property will make this Source scrobble to all Clients.
+      "name": "mySpotifySource",  // optional, friendly name for the log
+      "data": {                   // required, the data for your config
+        "clientId": "example",
+        //...
+      }
+    }
+  ],
   "clients": [
     {
-      "type": "maloja", // client name
-      "data": "/someLocation/aFile.json" // location to look for json configuration in
-    },
-    {
-      "type": "maloja", // client name
-      "data": {
-        // client configuration can also be an object
+      "type": "maloja",           // required, Client type
+      "name": "myConfig",         // required, a name to identifier your Client
+      "data": {                   // required, the data for your config
+        "url": "http://example.com",
+        //...
       }
     },
-    "maloja" // client can also be just the type if the configuration is located in the default configuration directory
   ]
 }

config/lastfm.json.example

@@ -0,0 +1,13 @@
+[
+  {
+    "name": "myLastFm",                                          // required, a name to identify your Client
+    "data": {
+      "apiKey": "string",                                        // required, Lastfm api key
+      "secret": "string",                                        // required, Lastfm shared secret
+      "session": "string",                                       // optional, session id returned from a complete auth flow.
+                                                                 //   if not specified will be generated during authentication
+      "redirectUri": "http://localhost:9078/lastfm/callback"     // optional, if not different than this default
+                                                                 //   callback for auth. Must have "lastfm/callback" in the url somewhere
+    }
+  }
+]

config/maloja.json.example

@@ -1,4 +1,9 @@
-{
-  "url": "https://domain.tld", // the base url of your maloja installation
-  "apiKey": "string" // your maloja api key
-}
+[
+  {
+    "name": "myMaloja",            // required, a name to identify your Client
+    "data": {
+      "url": "https://domain.tld", // required, the base url of your maloja installation
+      "apiKey": "string"           // required, your maloja api key
+    }
+  }
+]

config/plex.json.example

@@ -0,0 +1,11 @@
+[
+  {
+    "name": "default",                                        // optional, friendly name for logs
+    "clients": [],                                            // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "user": ["username@gmail.com","anotherUser@gmail.com"], // optional, list of users to scrobble tracks for
+      "libraries": ["music","my podcasts"],                   // optional, list of libraries to scrobble tracks from
+      "servers": ["myServer","anotherServer"]                 // optional, list of servers to scrobble tracks from
+    }
+  }
+]

config/spotify.json.example

@@ -1,6 +1,14 @@
-{
-  "clientId": "string", // spotify client id
-  "clientSecret": "string", // spotify client secret
-  "redirectUri": "http://localhost:9078/callback", // optional, spotify callback url. useful if this installation is behind a reverse proxy
-  "callbackPath": "callback" // optional, if redirectUri does not end with /callback then must specify path here
-}
+[
+  {
+    "name": "default",                                // optional, friendly name for logs
+    "clients": [],                                    // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "clientId": "string",                           // spotify client id -- required if not providing access token
+      "clientSecret": "string",                       // spotify client secret -- required if not providing access token
+      "accessToken": "string",                        // spotify access token -- required if not providing client id/secret
+      "refreshToken": "string",                       // spotify refresh token -- recommended to provide if not providing client id/secret
+      "redirectUri": "http://localhost:9078/callback",// spotify redirect URI -- required only if not the default shown here. URI must end in "callback"
+      "interval": 60                                  // optional, how long to wait before calling spotify for new tracks
+    }
+  }
+]

config/subsonic.json.example

@@ -0,0 +1,10 @@
+[
+  {
+    "name": "default",                        // optional, friendly name for logs
+    "data": {
+      "url": "http://localhost:4040/airsonic",// required, the url you would visit to listen to music on the web
+      "user": "yourUser",                     // required, username to login with
+      "password": "yourPassword"              // required, password to login with
+    }
+  }
+]

config/tautulli.json.example

@@ -0,0 +1,11 @@
+[
+  {
+    "name": "default",                                        // optional, friendly name for logs
+    "clients": [],                                            // optional, list of scrobble clients (by config name) that this source should scrobble to. Using an empty list or not including this property will make this source scrobble to all clients.
+    "data": {
+      "user": ["username@gmail.com","anotherUser@gmail.com"], // optional, list of users to scrobble tracks for
+      "libraries": ["music","my podcasts"],                   // optional, list of libraries to scrobble tracks from
+      "servers": ["myServer","anotherServer"]                 // optional, list of servers to scrobble tracks from
+    }
+  }
+]

docs/configuration.md

@@ -0,0 +1,182 @@
+# General
+
+General configuration options. These must be set through environmental variables because they affect initial startup of
+the app. **These variables are also available to Docker containers.**
+
+| Environmental Variable | Required? |   Default    |                                        Description                                        |
+|----------------------------|-----------|--------------|-------------------------------------------------------------------------------------------|
+| `CONFIG_DIR`               |         No | `CWD/config` | Directory to look for all other configuration files                                       |
+| `LOG_PATH`                 |         No | `CWD/logs`   | If `false` no logs will be written. If `string` will be the directory logs are written to |
+| `PORT`                     |         No | 9078         | Port to run web server on                                                                 |
+
+**The app must have permission to write to `CONFIG_DIR` in order to store the current spotify access token.**
+
+
+# Sources and (Scrobble) Clients
+
+The app has two types of configurations:
+
+* **Sources** -- Where plays are parsed from
+* **Clients** -- Scrobble clients that plays are scrobbled to
+
+All configurations can be configured through:
+* environmental variables
+* individual **json** files for each source/client type found in the `CONFIG_DIR` directory IE `config/spotify.json`
+* or through the main `config.json` (also found in `CONFIG_DIR` directory) using the `clients` or `sources` property under `data`:
+
+```json5
+// in config.json
+{
+  //...
+  "sources": [
+    {
+      "name": "myConfig",
+      "type": "spotify",
+      "clients": [
+        "myMalojaClient"
+      ],
+      "data": {
+        "clientId": "anExample"
+        //...
+      }
+    }
+  ]
+}
+```
+
+See [config.json.example](../config/config.json.example) for a short example of this or check out [the kitchen sink example](kitchensink.md).
+
+### ENV-Based or JSON-Based?
+
+multi-scrobbler can be configured differently depending on how you will use it. See which use-case fits you the best and then use that approach when setting up each configuration:
+
+#### ENV-Based (Single User)
+
+* You are the only person for whom the application is scrobbling
+* You may have many sources (Plex, Spotify, Tautulli...) but you only have one of each type of source
+* You have only one scrobble client
+* **Easier for small setups. Difficult for larger, multi-sourced setups (may want to switch to json)**
+* **Will not work for multi-user setups**
+
+#### JSON-Based (Multi User)
+
+* You are a single user but want to set up many sources
+* You want to use multi-scrobbler to scrobble for yourself and others IE family, friends, etc.
+* You may have many of each type of **Source** you want to scrobble from IE 2x Plex accounts, 3x Spotify accounts, 1x
+  Funkwhale...
+* You have more than one scrobble **Client** you want to scrobble to IE multiple Maloja servers, one for each person
+* You want only some **Sources** to scrobble to some **Clients** IE Fred's Spotify account scrobbles to Fred's Maloja
+  server, but not Mary's Maloja server
+
+Note: While you may mix and match configuration approaches it is recommended to **only use ENV-based configs if you are
+doing everything in ENV-based configurations.**
+
+# Sources
+
+## [Spotify](https://www.spotify.com)
+
+To access your Spotify history you must [register an application](https://developer.spotify.com/dashboard) to get a
+Client ID/Secret. Make sure to also whitelist your redirect URI in the application settings.
+
+### ENV-Based
+
+| Environmental Variable     | Required? |            Default             |                    Description                     |
+|----------------------------|-----------|----------------------------------|----------------------------------------------------|
+| `SPOTIFY_CLIENT_ID`        | Yes       |                                  |                                                    |
+| `SPOTIFY_CLIENT_SECRET`    | Yes       |                                  |                                                    |
+| `SPOTIFY_ACCESS_TOKEN`     | No         |                                  | Must include either this token or client id/secret |
+| `SPOTIFY_REFRESH_TOKEN`    | No         |                                  | If using access token this is also recommended      |
+| `SPOTIFY_REDIRECT_URI`     | No         | `http://localhost:{PORT}/callback` | URI must end in `callback`                         |
+
+### JSON-Based
+
+See [`spotify.json.example`](../config/spotify.json.example)
+
+## [Plex](https://plex.tv)
+
+Check the [instructions](plex.md) on how to setup a [webhooks](https://support.plex.tv/articles/115002267687-webhooks) to scrobble your plays.
+
+### ENV-Based
+
+| Environmental Variable | Required | Default |                   Description                   |
+|------------------------|----------|---------|-------------------------------------------------|
+| `PLEX_USER`              |        No |         | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
+
+### JSON-Based
+
+See [`plex.json.example`](../config/plex.json.example)
+
+## [Tautulli](https://tautulli.com)
+
+Check the [instructions](plex.md) on how to setup a notification agent.
+
+### ENV-Based
+
+| Environmental Variable | Required | Default |                   Description                   |
+|------------------------|----------|---------|-------------------------------------------------|
+| `TAUTULLI_USER`              |        No |         | The a comma-delimited list of usernames to scrobble tracks for. No usernames specified means all tracks by all users will be scrobbled. |
+
+### JSON-Based
+
+See [`tautulli.json.example`](../config/tautulli.json.example)
+
+## [Subsonic](http://www.subsonic.org/)
+
+Can use this source for any application that implements the [Subsonic API](http://www.subsonic.org/pages/api.jsp) (such as [Airsonic](https://airsonic.github.io/))
+
+**Known Issues:**
+* "Time played at" is somewhat inaccurate since the api only reports "played X minutes ago" so...
+  * All scrobble times are therefore "on the minute" and you may experience occasional duplicate scrobbles
+  * "played X minutes ago" sometimes is also not reported correctly
+* Multiple artists are reported as one value and cannot be separated
+* If using [Airsonic Advanced](https://github.com/airsonic-advanced/airsonic-advanced) the password used (under **Credentials**) must be **Decodable**  
+
+### ENV-Based
+
+| Environmental Variable     | Required? |            Default             |                    Description                     |
+|----------------------------|-----------|----------------------------------|----------------------------------------------------|
+| `SUBSONIC_USER`        | Yes       |                                  |                                                    |
+| `SUBSONIC_PASSWORD`    | Yes       |                                  |                                                    |
+| `SUBSONIC_URL`     | Yes         |                                  | Base url of your subsonic-api server |
+
+### JSON-Based
+
+See [`subsonic.json.example`](../config/subsonic.json.example)
+
+# Clients
+
+## [Maloja](https://github.com/krateng/maloja)
+
+### ENV-Based
+
+| Environmental Variable | Required? | Default |          Description          |
+|----------------------------|-----------|---------|-------------------------------|
+| `MALOJA_URL`               | Yes       |         | Base URL of your installation |
+| `MALOJA_API_KEY`           | Yes       |         | Api Key                       |
+
+### JSON-Based
+
+See [`maloja.json.example`](../config/maloja.json.example)
+
+## [Last.fm](https://www.last.fm)
+
+[Register for an API account here.](https://www.last.fm/api/account/create)
+
+The Callback URL is actually specified by multi-scrobbler but to keep things consistent you should use
+```
+http://localhost:9078/lastfm/callback
+```
+or replace `localhost:9078` with your own base URL
+
+### ENV-Based
+
+| Environmental Variable | Required? | Default |          Description          |
+|----------------------------|-----------|---------|-------------------------------|
+| `LASTFM_API_KEY`           | Yes       |         | Api Key from your API Account |
+| `LASTFM_SECRET`            | Yes       |         | Shared secret from your API Account |
+| `LASTFM_REDIRECT_URI`      | No        | `http://localhost:{PORT}/lastfm/callback`        | Url to use for authentication. Must include `lastfm/callback` somewhere in it |
+| `LASTFM_SESSION`           | No        |         | Session id. Will be generated by authentication flow if not provided.                       |
+
+### JSON-Based
+
+See [`lastfm.json.example`](../config/lastfm.json.example)

docs/kitchensink.md

@@ -0,0 +1,228 @@
+# Example Config using all Possible Features
+
+Scenario: 
+
+* You want to scrobble plays for yourself (Foxx), Fred, and Mary
+* Each person has their own Maloja server
+* Each person has their own Spotify account
+* You have your own Airsonic (subsonic) server you to scrobble from
+* Mary has her own Last.fm account she also wants to scrobble to
+* Fred has his own Spotify application and provides you with just his access and refresh token because he doesn't trust you (wtf Fred)
+* Fred has a Plex server and wants to scrobble everything he plays
+* Mary uses Fred's Plex server but only wants to scrobble her plays from the `podcast` library
+* The three of you have a shared library on Plex called `party` that you only play when you are hanging out. You want plays from that library to be scrobbled to everyone's servers.
+
+### All-in-one Config
+
+Using just one config file located at `CONFIG_DIR/config.json`:
+
+```json5
+{
+  "sources": [
+    {
+      "type": "spotify",
+      "name": "foxxSpot",
+      "clients": ["foxxMaloja"],
+      "data": {
+        "clientId": "foxxSpotifyAppId", 
+        "clientSecret": "foxxSpotifyAppSecret",
+      }
+    },
+    {
+      "type": "spotify",
+      "name": "marySpot",
+      "clients": ["maryMaloja"],
+      "data": {
+        "clientId": "foxxSpotifyAppId", // only need one application, it can be used by all users of this multi-scrobbler instance 
+        "clientSecret": "foxxSpotifyAppSecret",
+      }
+    },
+    {
+      "type": "spotify",
+      "name": "fredSpot",
+      "clients": ["fredMaloja"],
+      "data": {
+        "accessToken": "fredsToken",
+        "refreshToken": "fredsRefreshToken",
+        "interval": 120, // he also wants a slower check interval because his application already has heavy api usage
+      }
+    },
+    {
+      "type": "plex",
+      "name": "fredPlex",
+      "clients": ["fredMaloja"],
+      "data": {
+        "user": ["fred@email.com"]
+      }
+    },
+    {
+      "type": "plex",
+      "name": "maryPlex",
+      "clients": ["maryMaloja"],
+      "data": {
+        "user": ["mary@email.com"], // still need to specify mary as user so not all users who play from 'podcasts' get scrobbled
+        "libraries": ["podcasts"]
+      }
+    },
+    {
+      "type": "plex",
+      "name": "partyPlex",
+      // omitting clients (or making it empty) will make this Source scrobble to all Clients
+      "data": {
+        "libraries": ["party"],
+      }
+    },
+    {
+      "type": "subsonic",
+      "name": "foxxAirsonic",
+      "clients": ["foxxMaloja"],
+      "data": {
+        "user": "foxx",
+        "password": "foxxPassword",
+        "url": "https://airsonic.foxx.example"
+      }
+    },
+  ],
+  "clients": [
+    {
+      "type": "maloja",
+      "name": "foxxMaloja",
+      "data": {
+        "url": "https://maloja.foxx.example",
+        "apiKey": "foxxApiKey"
+      }
+    },
+    {
+      "type": "maloja",
+      "name": "fredMaloja",
+      "data": {
+        "url": "https://maloja.fred.example",
+        "apiKey": "fredApiKey"
+      }
+    },
+    {
+      "type": "maloja",
+      "name": "maryMaloja",
+      "data": {
+        "url": "https://maloja.mary.example",
+        "apiKey": "maryApiKey"
+      }
+    },
+    {
+      "type": "lastfm",
+      "name": "maryLFM",
+      "data": {
+        "apiKey": "maryApiKey",
+        "secret": "marySecret",
+      }
+    }
+  ]
+}
+
+```
+
+### Separate JSON files
+
+In `CONFIG_DIR/spotify.json`:
+
+```json5
+[
+  {
+    // may omit 'type' property since app knows this is file is for spotify configs
+    "name": "foxxSpot",
+    "clients": ["foxxMaloja"],
+    "data": {
+      "clientId": "foxxSpotifyAppId",
+      "clientSecret": "foxxSpotifyAppSecret",
+    }
+  },
+  {
+    "name": "marySpot",
+    "clients": ["maryMaloja"],
+    "data": {
+      "clientId": "foxxSpotifyAppId",
+      "clientSecret": "foxxSpotifyAppSecret",
+    }
+  },
+  {
+    "name": "fredSpot",
+    "clients": ["fredMaloja"],
+    "data": {
+      "accessToken": "fredsToken",
+      "refreshToken": "fredsRefreshToken",
+      "interval": 120,
+    }
+  },
+]
+```
+
+In `CONFIG_DIR/plex.json`
+
+```json5
+[
+  {
+    "name": "fredPlex",
+    "clients": ["fredMaloja"],
+    "data": {
+      "user": ["fred@email.com"]
+    }
+  },
+  {
+    "name": "maryPlex",
+    "clients": ["maryMaloja"],
+    "data": {
+      "user": ["mary@email.com"],
+      "libraries": ["podcasts"]
+    }
+  },
+  {
+    "name": "partyPlex",
+    "data": {
+      "libraries": ["party"],
+    }
+  }
+]
+```
+
+In `CONFIG_DIR/maloja.json`:
+
+```json5
+[
+  {
+    "name": "foxxMaloja",
+    "data": {
+      "url": "https://maloja.foxx.example",
+      "apiKey": "foxxApiKey"
+    }
+  },
+  {
+    "name": "fredMaloja",
+    "data": {
+      "url": "https://maloja.fred.example",
+      "apiKey": "fredApiKey"
+    }
+  },
+  {
+    "name": "maryMaloja",
+    "data": {
+      "url": "https://maloja.mary.example",
+      "apiKey": "maryApiKey"
+    }
+  }
+]
+```
+
+In `CONFIG_DIR/lastfm.json`:
+
+```json5
+[
+  {
+    "name": "maryLFM",
+    "data": {
+      "apiKey": "maryApiKey",
+      "secret": "marySecret",
+    }
+  }
+]
+```
+

docs/plex.md

@@ -0,0 +1,59 @@
+Tracks played on [Plex](https://plex.tv/) can be scrobbled either by:
+ * A [Tautulli](https://tautulli.com/) notification agent with a webhook.
+ * Using Plex [Webhooks](https://support.plex.tv/articles/115002267687-webhooks) (restricted to Plex Pass users)
+
+# Using Tautulli
+
+## Create a new Notification Agent
+
+* Navigate to the **Notification Agents** page in **Settings**
+* Click **Add a new notification agent**
+* Select **Webhook**
+
+## Configure the Agent
+
+The below sections correspond with the tabs available in the notification agent configuration popup.
+
+### Configuration
+* Webhook URL -- `http://localhost:9078/tautulli` (substitute your domain if different than the default)
+* Webhook Method -- POST
+
+### Triggers
+
+Select **Watched**
+
+### Conditions
+
+Refer to [Tautulli's documentation](https://github.com/Tautulli/Tautulli-Wiki/wiki/Custom-Notification-Conditions) if you need help here. It may be a good idea to restrict notifications to only one library (if you have a Music library, for instance)
+
+**This app will only scrobble an item if `media_type` is a "track", which is the default for all music.**
+
+### Data
+
+Expand the **Watched** dropdown and add the following code block to the **JSON Data** text field:
+
+```
+{
+"artist_name": "{artist_name}",
+"track_name": "{track_name}",
+"track_artist": "{track_artist}",
+"album_name": "{album_name}",
+"media_type": "{media_type}",
+"title":  "{title}",
+"duration": "{duration}",
+"username": "{username}"
+}
+```
+
+**Click the Save button to finish.**
+
+Your agent is now configured and ready to scrobble.
+
+# Using Plex Webhooks
+
+* Navigate to your **Account/Settings** and find the **Webhooks** page
+* Click **Add Webhook**
+* URL -- `http://localhost:9078/plex` (substitute your domain if different than the default)
+* **Save Changes**
+
+Plex is now configured to scrobble.

index.js

@@ -1,316 +1,424 @@
-import fs from "fs";
-import {addAsync} from '@awaitjs/express';
+import {addAsync, Router} from '@awaitjs/express';
 import express from 'express';
+import bodyParser from 'body-parser';
+import multer from 'multer';
 import winston from 'winston';
+import dayjs from 'dayjs';
+import utc from 'dayjs/plugin/utc.js';
+import isBetween from 'dayjs/plugin/isBetween.js';
+import relativeTime from 'dayjs/plugin/relativeTime.js';
 import {Writable} from 'stream';
 import 'winston-daily-rotate-file';
-import {readJson, sleep, writeFile, buildTrackString} from "./utils.js";
-import SpotifyWebApi from "spotify-web-api-node";
-import MalojaScrobbler from "./clients/MalojaScrobbler.js";
+import {
+    buildTrackString,
+    capitalize,
+    labelledFormat,
+    longestString,
+    readJson, sleep,
+    truncateStringToLength
+} from "./utils.js";
+import Clients from './clients/ScrobbleClients.js';
+import ScrobbleSources from "./sources/ScrobbleSources.js";
+import {makeClientCheckMiddle, makeSourceCheckMiddle} from "./server/middleware.js";
+import TautulliSource from "./sources/TautulliSource.js";
+import PlexSource from "./sources/PlexSource.js";
 
-const {format, createLogger, transports} = winston;
-const {combine, printf, timestamp} = format;
+const storage = multer.memoryStorage()
+const upload = multer({storage: storage})
+
+dayjs.extend(utc)
+dayjs.extend(isBetween);
+dayjs.extend(relativeTime);
+
+const {transports} = winston;
 
 let output = []
 const stream = new Writable()
 stream._write = (chunk, encoding, next) => {
-    output.unshift(chunk.toString());
-    output = output.slice(0, 51);
+    output.unshift(chunk.toString().replace('\n', ''));
+    output = output.slice(0, 101);
     next()
 }
-const streamTransport = new winston.transports.Stream({stream})
+const streamTransport = new winston.transports.Stream({
+    stream,
+})
 
-const myFormat = printf(({level, message, label = 'App', timestamp}) => {
-    return `${timestamp} [${label}] ${level}: ${message}`;
+const logConfig = {
+    level: process.env.LOG_LEVEL || 'info',
+    sort: 'descending',
+    limit: 50,
+}
+
+const availableLevels = ['info', 'debug'];
+const logPath = process.env.LOG_DIR || `${process.cwd()}/logs`;
+const port = process.env.PORT ?? 9078;
+const localUrl = `http://localhost:${port}`;
+
+const rotateTransport = new winston.transports.DailyRotateFile({
+    dirname: logPath,
+    createSymlink: true,
+    symlinkName: 'scrobble-current.log',
+    filename: 'scrobble-%DATE%.log',
+    datePattern: 'YYYY-MM-DD',
+    maxSize: '5m'
 });
 
-const logger = createLogger({
-    level: 'debug',
-    format: combine(
-        timestamp(),
-        myFormat
-    ),
-    transports: [
-        new transports.Console(),
-        streamTransport,
-    ]
-});
+const consoleTransport = new transports.Console();
 
-const scopes = ['user-read-recently-played', 'user-read-currently-playing'];
-const state = 'random';
-let lastTrackPlayedAt = new Date();
+const myTransports = [
+    consoleTransport,
+    streamTransport,
+];
+
+if (typeof logPath === 'string') {
+    myTransports.push(rotateTransport);
+}
+
+const loggerOptions = {
+    level: logConfig.level,
+    format: labelledFormat(),
+    transports: myTransports,
+};
+
+winston.loggers.add('default', loggerOptions);
+
+const logger = winston.loggers.get('default');
 
 const configDir = process.env.CONFIG_DIR || `${process.cwd()}/config`;
-const configLocation = process.env.CONFIG_PATH || `${configDir}/config.json`;
 
 const app = addAsync(express());
+const router = Router();
 
-try {
-    (async function () {
-
-        let spotifyAsyncFunc = null;
+app.use(router);
+app.use(bodyParser.json());
 
+(async function () {
+    try {
         // try to read a configuration file
         let config = {};
         try {
-            config = await readJson(configLocation);
+            config = await readJson(`${configDir}/config.json`, {throwOnNotFound: false});
         } catch (e) {
-            logger.warn('Could not read config file');
-            logger.error(e);
+            logger.warn('App config file exists but could not be parsed!');
         }
 
         // setup defaults for other configs and general config
         const {
-            logPath: logPathRaw = process.env.LOG_PATH || true,
-            interval = 60,
-            port = process.env.PORT ?? 9078,
-            spotify: spotifyConfigRaw = process.env.SPOTIFY_CONFIG_PATH || `${configDir}/spotify.json`,
-            clients = [],
+            spotify,
+            plex,
         } = config || {};
 
-        const localUrl = `http://localhost:${port}`;
+        /*
+        * setup clients
+        * */
+        const scrobbleClients = new Clients(configDir);
+        await scrobbleClients.buildClientsFromConfig();
+        if (scrobbleClients.clients.length === 0) {
+            logger.warn('No scrobble clients were configured!')
+        }
 
-        // first thing, if user wants to log to file set it up now
-        if (logPathRaw !== false) {
-            let logPath = `${process.cwd()}/logs`;
-            if (typeof logPathRaw === 'string') {
-                logPath = logPathRaw;
+        const scrobbleSources = new ScrobbleSources(localUrl, configDir);
+        let deprecatedConfigs = [];
+        if (spotify !== undefined) {
+            logger.warn(`Using 'spotify' top-level property in config.json is deprecated and will be removed in next major version. Please use 'sources' instead.`)
+            deprecatedConfigs.push({
+                type: 'spotify',
+                name: 'unnamed',
+                source: 'config.json (top level)',
+                mode: 'single',
+                data: spotify
+            });
+        }
+        if (plex !== undefined) {
+            logger.warn(`Using 'plex' top-level property in config.json is deprecated and will be removed in next major version. Please use 'sources' instead.`)
+            deprecatedConfigs.push({
+                type: 'plex',
+                name: 'unnamed',
+                source: 'config.json (top level)',
+                mode: 'single',
+                data: plex
+            });
+        }
+        await scrobbleSources.buildSourcesFromConfig(deprecatedConfigs);
+
+        const clientCheckMiddle = makeClientCheckMiddle(scrobbleClients);
+        const sourceCheckMiddle = makeSourceCheckMiddle(scrobbleSources);
+
+        app.getAsync('/', async function (req, res) {
+            let slicedLog = output.slice(0, logConfig.limit + 1);
+            if (logConfig.sort === 'ascending') {
+                slicedLog.reverse();
             }
-            logger.add(new winston.transports.DailyRotateFile({
-                level: 'info', // don't need to add a bunch of noise to files
-                dirname: logPath,
-                createSymlink: true,
-                symlinkName: 'scrobble-current.log',
-                filename: 'scrobble-%DATE%.log',
-                datePattern: 'YYYY-MM-DD',
-                maxSize: '5m'
-            }))
-        }
+            const sourceData = scrobbleSources.sources.map((x) => {
+                const {type, tracksDiscovered = 0, name, canPoll = false, polling = false} = x;
+                const base = {type, display: capitalize(type), tracksDiscovered, name, canPoll, hasAuth: false};
+                if (canPoll) {
+                    base.status = polling ? 'Running' : 'Idle';
+                } else {
+                    base.status = tracksDiscovered > 0 ? 'Received Data' : 'Awaiting Data'
+                }
+                switch (x.type) {
+                    case 'spotify':
+                        const authed = x.spotifyApi === undefined || x.spotifyApi.getAccessToken() !== undefined;
+                        return {
+                            ...base,
+                            hasAuth: true,
+                            authed,
+                            status: authed ? base.status : 'Auth Interaction Required',
+                        }
+                    default:
+                        return base;
+                }
+            });
+            const clientData = scrobbleClients.clients.map((x) => {
+                const {type, tracksScrobbled = 0, name} = x;
+                const base = {
+                    type,
+                    display: capitalize(type),
+                    tracksDiscovered: tracksScrobbled,
+                    name,
+                    hasAuth: false,
+                    status: tracksScrobbled > 0 ? 'Received Data' : 'Awaiting Data'
+                };
+                switch (x.type) {
+                    case 'lastfm':
+                        const authed = x.initialized;
+                        return {
+                            ...base,
+                            hasAuth: true,
+                            authed,
+                            status: authed ? base.status : 'Auth Interaction Required',
+                        }
+                    default:
+                        return base;
+                }
+            })
+            res.render('status', {
+                sources: sourceData,
+                clients: clientData,
+                logs: {
+                    output: slicedLog,
+                    limit: [10, 20, 50, 100].map(x => `<a class="capitalize ${logConfig.limit === x ? 'bold' : ''}" href="logs/settings/update?limit=${x}">${x}</a>`).join(' | '),
+                    sort: ['ascending', 'descending'].map(x => `<a class="capitalize ${logConfig.sort === x ? 'bold' : ''}" href="logs/settings/update?sort=${x}">${x}</a>`).join(' | '),
+                    level: availableLevels.map(x => `<a class="capitalize ${logConfig.level === x ? 'bold' : ''}" href="logs/settings/update?level=${x}">${x}</a>`).join(' | ')
+                }
+            });
+        })
 
-        if (interval < 15) {
-            console.warn('Interval should be above 30 seconds...😬');
-        }
-
-        let spotifyCreds = {};
-        try {
-            spotifyCreds = await readJson('./spotifyCreds.json');
-        } catch (e) {
-            logger.warn('Current spotify access token was not parsable or file does not exist (this could be normal)');
-        }
-
-        let spotifyConfig = spotifyConfigRaw;
-        if (typeof spotifyConfigRaw === 'string') {
-            try {
-                spotifyConfig = await readJson(spotifyConfigRaw);
-            } catch (e) {
-                logger.warn('Could not read spotify config file');
-                logger.error(e);
+        app.postAsync('/tautulli', async function (req, res) {
+            const payload = TautulliSource.formatPlayObj(req.body, true);
+            // try to get config name from payload
+            if (req.body.scrobblerConfig !== undefined) {
+                const source = scrobbleSources.getByName(req.body.scrobblerConfig);
+                if (source !== undefined) {
+                    if (source.type !== 'tautulli') {
+                        this.logger.warn(`Tautulli event specified a config name but the configured source was not a Tautulli type: ${req.body.scrobblerConfig}`);
+                        return res.send('OK');
+                    } else {
+                        await source.handle(payload, scrobbleClients);
+                        return res.send('OK');
+                    }
+                } else {
+                    this.logger.warn(`Tautulli event specified a config name but no configured source found: ${req.body.scrobblerConfig}`);
+                    return res.send('OK');
+                }
             }
-        }
-
-        const {
-            accessToken = process.env.SPOTIFY_ACCESS_TOKEN,
-            clientId = process.env.SPOTIFY_CLIENT_ID,
-            clientSecret = process.env.SPOTIFY_CLIENT_SECRET,
-            redirectUri = process.env.SPOTIFY_REDIRECT_URI,
-            refreshToken = process.env.SPOTIFY_REFRESH_TOKEN,
-        } = spotifyConfig;
-
-        const rdUri = redirectUri || `${localUrl}/callback`;
-
-
-        const {token = accessToken, refreshToken: rt = refreshToken} = spotifyCreds;
-
-        if (token === undefined) {
-            if (clientId === undefined) {
-                throw new Error('ClientId not defined');
+            // if none specified we'll iterate through all tautulli sources and hopefully the user has configured them with filters
+            const tSources = scrobbleSources.getByType('tautulli');
+            for (const source of tSources) {
+                await source.handle(payload, scrobbleClients);
             }
-            if (clientSecret === undefined) {
-                throw new Error('ClientSecret not defined');
-            }
-        }
 
-        const spotifyApi = new SpotifyWebApi({
-            clientId,
-            clientSecret,
-            accessToken: token,
-            redirectUri: rdUri,
-            refreshToken: rt,
-        });
-
-        const scrobbleClients = await createClients(clients, configDir);
-        if (scrobbleClients.length === 0) {
-            throw new Error('No scrobble clients were configured');
-        }
-
-
-        app.getAsync('/authSpotify', async function (req, res) {
-            logger.info('Redirecting to spotify authorization url');
-            res.redirect(spotifyApi.createAuthorizeURL(scopes, state));
-        });
-
-        app.postAsync('/pollSpotify', async function (req, res) {
-            spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients);
             res.send('OK');
         });
 
-        app.getAsync(`/callback`, async function (req, res, next) {
-            const {error, code} = req.query;
-            if (error === undefined) {
-                const tokenResponse = await spotifyApi.authorizationCodeGrant(code);
-                spotifyApi.setAccessToken(tokenResponse.body['access_token']);
-                spotifyApi.setRefreshToken(tokenResponse.body['refresh_token']);
-                await writeFile('spotifyCreds.json', JSON.stringify({
-                    token: tokenResponse.body['access_token'],
-                    refreshToken: tokenResponse.body['refresh_token']
-                }));
-                logger.info('Got auth code from callback!');
-                spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients);
-                return res.send('OK');
-            } else {
-                throw new Error('User denied oauth access');
+        app.postAsync('/plex', upload.any(), async function (req, res) {
+            const {
+                body: {
+                    payload
+                } = {}
+            } = req;
+            if (payload !== undefined) {
+                const playObj = PlexSource.formatPlayObj(JSON.parse(payload), true);
+
+                const pSources = scrobbleSources.getByType('plex');
+                for (const source of pSources) {
+                    await source.handle(playObj, scrobbleClients);
+                }
+            }
+            res.send('OK');
+        });
+
+        app.use('/client/auth', clientCheckMiddle);
+        app.getAsync('/client/auth', async function (req, res) {
+            const {
+                scrobbleClient,
+            } = req;
+
+            switch (scrobbleClient.type) {
+                case 'lastfm':
+                    res.redirect(scrobbleClient.getAuthUrl());
+                    break;
+                default:
+                    return res.status(400).send(`Specified client does not have auth implemented (${scrobbleClient.type})`);
             }
         });
 
-        if (token === undefined) {
-            logger.info('No access token found');
-            logger.info(`Open ${localUrl}/authSpotify to continue`);
-        } else {
-            spotifyAsyncFunc = pollSpotify(spotifyApi, interval, scrobbleClients)
-            logger.info(`Server started at ${localUrl}`);
-        }
+        app.use('/source/auth', sourceCheckMiddle);
+        app.getAsync('/source/auth', async function (req, res) {
+            const {
+                scrobbleSource: source,
+                sourceName: name,
+            } = req;
 
-        const server = await app.listen(port)
-    }());
-} catch (e) {
-    logger.error('Exited with uncaught error');
-    logger.error(e);
-}
-
-const pollSpotify = async function (spotifyApi, interval = 60, clients = []) {
-    logger.info('Starting spotify polling', {label: 'Spotify'});
-    try {
-        while (true) {
-            let data = {};
-            logger.debug('Refreshing recently played', {label: 'Spotify'})
-            try {
-                data = await spotifyApi.getMyRecentlyPlayedTracks({
-                    limit: 20
-                });
-            } catch (e) {
-                if (e.statusCode === 401) {
-                    logger.info('Access token was not valid, attempting to refresh', {label: 'Spotify'});
-                    const tokenResponse = await spotifyApi.refreshAccessToken();
-                    const {body: {
-                        access_token,
-                        // spotify may return a new refresh token
-                        // if it doesn't then continue to use the last refresh token we received
-                        refresh_token = spotifyApi.getRefreshToken(),
-                    } = {}} = tokenResponse;
-                    spotifyApi.setAccessToken(access_token);
-                    await writeFile('spotifyCreds.json', JSON.stringify({
-                        token: access_token,
-                        refreshToken: refresh_token,
-                    }));
-                    data = await spotifyApi.getMyRecentlyPlayedTracks({
-                        limit: 20
-                    });
-                } else {
-                    throw e;
-                }
-            }
-            let newLastPLayedAt = undefined;
-            const now = new Date();
-            for (const playObj of data.body.items) {
-                const {track: {name: trackName, duration_ms }, played_at} = playObj;
-                const playDate = new Date(played_at);
-                // compare play time to most recent track played_at scrobble
-                if (playDate.getTime() > lastTrackPlayedAt.getTime()) {
-                    logger.info(`New Track => ${buildTrackString(playObj)}`, {label: 'Spotify'});
-                    // so we always get just the most recent played_at
-                    if (newLastPLayedAt === undefined) {
-                        newLastPLayedAt = playDate;
+            switch (source.type) {
+                case 'spotify':
+                    if (source.spotifyApi === undefined) {
+                        res.status(400).send('Spotify configuration is not valid');
+                    } else {
+                        logger.info('Redirecting to spotify authorization url');
+                        res.redirect(source.createAuthUrl());
                     }
-                    const closeToInterval = Math.abs(now.getTime() - playDate.getTime()) / 1000 < 5;
-                    if (closeToInterval) {
-                        // because the interval check was so close to the play date we are going to delay client calls for a few secs
-                        // this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
-                        // additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
-                        logger.info('Track is close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first', {label: 'Spotify'});
-                        await sleep(10 * 1000);
-                    }
-                    for (const client of clients) {
-                        if (closeToInterval || client.scrobblesLastCheckedAt().getTime() < now.getTime()) {
-                            await client.refreshScrobbles();
-                        }
-                        if (!client.alreadyScrobbled(trackName, playDate, duration_ms / 1000)) {
-                            await client.scrobble(playObj);
-                        }
-                    }
-                } else {
                     break;
+                default:
+                    return res.status(400).send(`Specified source does not have auth implemented (${source.type})`);
+            }
+        });
+
+        app.use('/poll', sourceCheckMiddle);
+        app.getAsync('/poll', async function (req, res) {
+            const {
+                scrobbleSource: source,
+            } = req;
+
+            if (!source.canPoll) {
+                return res.status(400).send(`Specified source cannot poll (${source.type})`);
+            }
+
+            source.poll(scrobbleClients);
+            res.send('OK');
+        });
+
+        app.use('/recent', sourceCheckMiddle);
+        app.getAsync('/recent', async function (req, res) {
+            const {
+                scrobbleSource: source,
+            } = req;
+            if (!source.canPoll) {
+                return res.status(400).send(`Specified source cannot retrieve recent plays (${source.type})`);
+            }
+
+            const result = await source.getRecentlyPlayed({formatted: true});
+            const artistTruncFunc = truncateStringToLength(Math.min(40, longestString(result.map(x => x.data.artists.join(' / ')).flat())));
+            const trackLength = longestString(result.map(x => x.data.track))
+            const plays = result.map((x) => {
+                const {
+                    meta: {
+                        url: {
+                            web
+                        } = {}
+                    } = {}
+                } = x;
+                const buildOpts = {
+                    include: ['time', 'timeFromNow', 'track', 'artist'],
+                    transformers: {
+                        artists: a => artistTruncFunc(a.join(' / ')).padEnd(33),
+                        track: t => t.padEnd(trackLength)
+                    }
                 }
-                if (newLastPLayedAt !== undefined) {
-                    lastTrackPlayedAt = newLastPLayedAt;
+                if (web !== undefined) {
+                    buildOpts.transformers.track = t => `<a href="${web}">${t}</a>${''.padEnd(Math.max(trackLength - t.length, 0))}`;
+                }
+                return buildTrackString(x, buildOpts);
+            });
+            res.render('recent', {plays, name: source.name, sourceType: source.type});
+        });
+
+        app.getAsync('/logs/settings/update', async function (req, res) {
+            const e = req.query;
+            for (const [setting, val] of Object.entries(req.query)) {
+                switch (setting) {
+                    case 'limit':
+                        logConfig.limit = Number.parseInt(val);
+                        break;
+                    case 'sort':
+                        logConfig.sort = val;
+                        break;
+                    case 'level':
+                        logConfig.level = val;
+                        for (const [key, logger] of winston.loggers.loggers) {
+                            logger.level = val;
+                        }
+                        break;
                 }
             }
-            // sleep for interval
-            logger.debug(`Sleeping for interval (${interval}s)`, {label: 'Spotify'});
-            await sleep(interval * 1000);
-        }
-    } catch (e) {
-        logger.error('Error occurred while in spotify polling loop', {label: 'Spotify'});
-        logger.error(e, {label: 'Spotify'});
-    }
-};
+            res.send('OK');
+        });
 
-
-const createClients = async function (clientConfigs = [], configDir = '.') {
-    const clients = [];
-    for (const config of clientConfigs) {
-        const dataType = typeof config;
-        if (!['object', 'string'].includes(dataType)) {
-            throw new Error('All client configs must be objects or strings');
-        }
-        const clientType = dataType === 'string' ? config : config.type;
-        switch (clientType) {
-            case 'maloja':
-                let data = `${configDir}/maloja.json`;
-                if (dataType === 'object') {
-                    const {data: dataProp = process.env.MALOJA_CONFIG_PATH || `${configDir}/maloja.json`} = config;
-                    data = dataProp;
-                }
-                let malojaConfig;
-                if (typeof data === 'string') {
-                    try {
-                        malojaConfig = await readJson(data);
-                    } catch (e) {
-                        logger.warn('Maloja config was not parsable or file does not exist');
-                    }
-                } else {
-                    malojaConfig = data;
-                }
+        app.getAsync(/.*callback$/, async function (req, res) {
+            const {
+                query: {
+                    state
+                } = {}
+            } = req;
+            if (req.url.includes('lastfm')) {
                 const {
-                    url = process.env.MALOJA_URL,
-                    apiKey = process.env.MALOJA_API_KEY
-                } = malojaConfig;
-                if (url === undefined && apiKey === undefined) {
-                    // the user probably didn't set anything up for this client at all, don't log
-                    continue;
+                    query: {
+                        token
+                    } = {}
+                } = req;
+                const client = scrobbleClients.getByName(state);
+                try {
+                    await client.authenticate(token);
+                    await client.initialize();
+                    return res.send('OK');
+                } catch (e) {
+                    return res.send(e.message);
                 }
-                if (url === undefined) {
-                    logger.warn('Maloja url not found in config');
-                    continue;
+            } else {
+                logger.info('Received auth code callback from Spotify', {label: 'Spotify'});
+                const source = scrobbleSources.getByName(state);
+                const tokenResult = await source.handleAuthCodeCallback(req.query);
+                let responseContent = 'OK';
+                if (tokenResult === true) {
+                    source.poll(scrobbleClients);
+                } else {
+                    responseContent = tokenResult;
                 }
-                if (apiKey === undefined) {
-                    logger.warn('Maloja api key not found in config');
-                    continue;
-                }
-                clients.push(new MalojaScrobbler(logger, malojaConfig));
-                break;
-            default:
-                break;
+                return res.send(responseContent);
+            }
+        });
+
+        let anyNotReady = false;
+        for (const source of scrobbleSources.sources.filter(x => x.canPoll === true)) {
+            await sleep(1500); // stagger polling by 1.5 seconds so that log messages for each source don't get mixed up
+            switch (source.type) {
+                case 'spotify':
+                    if (source.spotifyApi !== undefined) {
+                        if (source.spotifyApi.getAccessToken() === undefined) {
+                            anyNotReady = true;
+                        } else {
+                            source.poll(scrobbleClients);
+                        }
+                    }
+                    break;
+                default:
+                    if (source.poll !== undefined) {
+                        source.poll(scrobbleClients);
+                    }
+            }
         }
+        if (anyNotReady) {
+            logger.info(`Some sources are not ready, open ${localUrl} to continue`);
+        }
+
+        app.set('views', './views');
+        app.set('view engine', 'ejs');
+        logger.info(`Server started at ${localUrl}`);
+        const server = await app.listen(port)
+    } catch (e) {
+        logger.error('Exited with uncaught error');
+        logger.error(e);
     }
-    return clients;
-}
+}());
+

package-lock.json

@@ -28,6 +28,19 @@
         "negotiator": "0.6.2"
       }
     },
+    "ansi-styles": {
+      "version": "3.2.1",
+      "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz",
+      "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==",
+      "requires": {
+        "color-convert": "^1.9.0"
+      }
+    },
+    "append-field": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/append-field/-/append-field-1.0.0.tgz",
+      "integrity": "sha1-HjRA6RXwsSA9I3SOeO3XubW0PlY="
+    },
     "array-flatten": {
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
@@ -43,6 +56,11 @@
       "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
       "integrity": "sha1-x57Zf380y48robyXkLzDZkdLS3k="
     },
+    "balanced-match": {
+      "version": "1.0.0",
+      "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.0.tgz",
+      "integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c="
+    },
     "body-parser": {
       "version": "1.19.0",
       "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.19.0.tgz",
@@ -80,11 +98,67 @@
         }
       }
     },
+    "brace-expansion": {
+      "version": "1.1.11",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
+      "integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
+      "requires": {
+        "balanced-match": "^1.0.0",
+        "concat-map": "0.0.1"
+      }
+    },
+    "buffer-from": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.1.tgz",
+      "integrity": "sha512-MQcXEUbCKtEo7bhqEs6560Hyd4XaovZlO/k9V3hjVUF/zwW7KBVdSK4gIt/bzwS9MbR5qob+F5jusZsb0YQK2A=="
+    },
+    "busboy": {
+      "version": "0.2.14",
+      "resolved": "https://registry.npmjs.org/busboy/-/busboy-0.2.14.tgz",
+      "integrity": "sha1-bCpiLvz0fFe7vh4qnDetNseSVFM=",
+      "requires": {
+        "dicer": "0.2.5",
+        "readable-stream": "1.1.x"
+      },
+      "dependencies": {
+        "isarray": {
+          "version": "0.0.1",
+          "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz",
+          "integrity": "sha1-ihis/Kmo9Bd+Cav8YDiTmwXR7t8="
+        },
+        "readable-stream": {
+          "version": "1.1.14",
+          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz",
+          "integrity": "sha1-fPTFTvZI44EwhMY23SB54WbAgdk=",
+          "requires": {
+            "core-util-is": "~1.0.0",
+            "inherits": "~2.0.1",
+            "isarray": "0.0.1",
+            "string_decoder": "~0.10.x"
+          }
+        },
+        "string_decoder": {
+          "version": "0.10.31",
+          "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz",
+          "integrity": "sha1-YuIDvEF2bGwoyfyEMB2rHFMQ+pQ="
+        }
+      }
+    },
     "bytes": {
       "version": "3.1.0",
       "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.0.tgz",
       "integrity": "sha512-zauLjrfCG+xvoyaqLoV8bLVXXNGC4JqlxFCutSDWA6fJrTo2ZuvLYTqZ7aHBLZSMOopbzwv8f+wZcVzfVTI2Dg=="
     },
+    "chalk": {
+      "version": "2.4.2",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.2.tgz",
+      "integrity": "sha512-Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==",
+      "requires": {
+        "ansi-styles": "^3.2.1",
+        "escape-string-regexp": "^1.0.5",
+        "supports-color": "^5.3.0"
+      }
+    },
     "color": {
       "version": "3.0.0",
       "resolved": "https://registry.npmjs.org/color/-/color-3.0.0.tgz",
@@ -143,6 +217,51 @@
       "resolved": "https://registry.npmjs.org/component-emitter/-/component-emitter-1.3.0.tgz",
       "integrity": "sha512-Rd3se6QB+sO1TwqZjscQrurpEPIfO0/yYnSin6Q/rD3mOutHvUrCAhJub3r90uNb+SESBuE0QYoB90YdfatsRg=="
     },
+    "concat-map": {
+      "version": "0.0.1",
+      "resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
+      "integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s="
+    },
+    "concat-stream": {
+      "version": "1.6.2",
+      "resolved": "https://registry.npmjs.org/concat-stream/-/concat-stream-1.6.2.tgz",
+      "integrity": "sha512-27HBghJxjiZtIk3Ycvn/4kbJk/1uZuJFfuPEns6LaEvpvG1f0hTea8lilrouyo9mVc2GWdcEZ8OLoGmSADlrCw==",
+      "requires": {
+        "buffer-from": "^1.0.0",
+        "inherits": "^2.0.3",
+        "readable-stream": "^2.2.2",
+        "typedarray": "^0.0.6"
+      },
+      "dependencies": {
+        "readable-stream": {
+          "version": "2.3.7",
+          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.7.tgz",
+          "integrity": "sha512-Ebho8K4jIbHAxnuxi7o42OrZgF/ZTNcsZj6nRKyUmkhLFq8CHItp/fy6hQZuZmP/n3yZ9VBUbp4zz/mX8hmYPw==",
+          "requires": {
+            "core-util-is": "~1.0.0",
+            "inherits": "~2.0.3",
+            "isarray": "~1.0.0",
+            "process-nextick-args": "~2.0.0",
+            "safe-buffer": "~5.1.1",
+            "string_decoder": "~1.1.1",
+            "util-deprecate": "~1.0.1"
+          }
+        },
+        "safe-buffer": {
+          "version": "5.1.2",
+          "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.2.tgz",
+          "integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="
+        },
+        "string_decoder": {
+          "version": "1.1.1",
+          "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz",
+          "integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==",
+          "requires": {
+            "safe-buffer": "~5.1.0"
+          }
+        }
+      }
+    },
     "content-disposition": {
       "version": "0.5.3",
       "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.3.tgz",
@@ -183,10 +302,10 @@
       "resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.2.tgz",
       "integrity": "sha1-tf1UIgqivFq1eqtxQMlAdUUDwac="
     },
-    "date-fns": {
-      "version": "2.16.1",
-      "resolved": "https://registry.npmjs.org/date-fns/-/date-fns-2.16.1.tgz",
-      "integrity": "sha512-sAJVKx/FqrLYHAQeN7VpJrPhagZc9R4ImZIWYRFZaaohR3KzmuK88touwsSwSVT8Qcbd4zoDsnGfX4GFB4imyQ=="
+    "dayjs": {
+      "version": "1.9.6",
+      "resolved": "https://registry.npmjs.org/dayjs/-/dayjs-1.9.6.tgz",
+      "integrity": "sha512-HngNLtPEBWRo8EFVmHFmSXAjtCX8rGNqeXQI0Gh7wCTSqwaKgPIDqu9m07wABVopNwzvOeCb+2711vQhDlcIXw=="
     },
     "debug": {
       "version": "4.2.0",
@@ -211,11 +330,51 @@
       "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz",
       "integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA="
     },
+    "dicer": {
+      "version": "0.2.5",
+      "resolved": "https://registry.npmjs.org/dicer/-/dicer-0.2.5.tgz",
+      "integrity": "sha1-WZbAhrszIYyBLAkL3cCc0S+stw8=",
+      "requires": {
+        "readable-stream": "1.1.x",
+        "streamsearch": "0.1.2"
+      },
+      "dependencies": {
+        "isarray": {
+          "version": "0.0.1",
+          "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz",
+          "integrity": "sha1-ihis/Kmo9Bd+Cav8YDiTmwXR7t8="
+        },
+        "readable-stream": {
+          "version": "1.1.14",
+          "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.1.14.tgz",
+          "integrity": "sha1-fPTFTvZI44EwhMY23SB54WbAgdk=",
+          "requires": {
+            "core-util-is": "~1.0.0",
+            "inherits": "~2.0.1",
+            "isarray": "0.0.1",
+            "string_decoder": "~0.10.x"
+          }
+        },
+        "string_decoder": {
+          "version": "0.10.31",
+          "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz",
+          "integrity": "sha1-YuIDvEF2bGwoyfyEMB2rHFMQ+pQ="
+        }
+      }
+    },
     "ee-first": {
       "version": "1.1.1",
       "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz",
       "integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0="
     },
+    "ejs": {
+      "version": "3.1.5",
+      "resolved": "https://registry.npmjs.org/ejs/-/ejs-3.1.5.tgz",
+      "integrity": "sha512-dldq3ZfFtgVTJMLjOe+/3sROTzALlL9E34V4/sDtUd/KlBSS0s6U1/+WPE1B4sj9CXHJpL1M6rhNJnc9Wbal9w==",
+      "requires": {
+        "jake": "^10.6.1"
+      }
+    },
     "enabled": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/enabled/-/enabled-2.0.0.tgz",
@@ -231,6 +390,11 @@
       "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz",
       "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg="
     },
+    "escape-string-regexp": {
+      "version": "1.0.5",
+      "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz",
+      "integrity": "sha1-G2HAViGQqN/2rjuyzwIAyhMLhtQ="
+    },
     "etag": {
       "version": "1.8.1",
       "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
@@ -316,6 +480,14 @@
         "moment": "^2.11.2"
       }
     },
+    "filelist": {
+      "version": "1.0.1",
+      "resolved": "https://registry.npmjs.org/filelist/-/filelist-1.0.1.tgz",
+      "integrity": "sha512-8zSK6Nu0DQIC08mUC46sWGXi+q3GGpKydAG36k+JDba6VRpkevvOWUW5a/PhShij4+vHT9M+ghgG7eM+a9JDUQ==",
+      "requires": {
+        "minimatch": "^3.0.4"
+      }
+    },
     "finalhandler": {
       "version": "1.1.2",
       "resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.1.2.tgz",
@@ -375,6 +547,11 @@
       "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz",
       "integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac="
     },
+    "has-flag": {
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz",
+      "integrity": "sha1-tdRU3CGZriJWmfNGfloH87lVuv0="
+    },
     "http-errors": {
       "version": "1.7.2",
       "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.7.2.tgz",
@@ -427,11 +604,34 @@
       "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
       "integrity": "sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE="
     },
+    "jake": {
+      "version": "10.8.2",
+      "resolved": "https://registry.npmjs.org/jake/-/jake-10.8.2.tgz",
+      "integrity": "sha512-eLpKyrfG3mzvGE2Du8VoPbeSkRry093+tyNjdYaBbJS9v17knImYGNXQCUV0gLxQtF82m3E8iRb/wdSQZLoq7A==",
+      "requires": {
+        "async": "0.9.x",
+        "chalk": "^2.4.2",
+        "filelist": "^1.0.1",
+        "minimatch": "^3.0.4"
+      },
+      "dependencies": {
+        "async": {
+          "version": "0.9.2",
+          "resolved": "https://registry.npmjs.org/async/-/async-0.9.2.tgz",
+          "integrity": "sha1-rqdNXmHB+JlhO/ZL2mbUx48v0X0="
+        }
+      }
+    },
     "kuler": {
       "version": "2.0.0",
       "resolved": "https://registry.npmjs.org/kuler/-/kuler-2.0.0.tgz",
       "integrity": "sha512-Xq9nH7KlWZmXAtodXDDRE7vs6DU1gTU8zYDHDiWLSip45Egwq3plLHzPn27NgvzL2r1LMPC1vdqh98sQxtqj4A=="
     },
+    "lastfm-node-client": {
+      "version": "2.2.0",
+      "resolved": "https://registry.npmjs.org/lastfm-node-client/-/lastfm-node-client-2.2.0.tgz",
+      "integrity": "sha512-nhHxrRPaNKIJnEuRov68HGjQRAFel2UWnG86ieQAklaVOdBciC6DO0vajZjzPZGqwvUAxQzDHPsp2ceDOtKW2w=="
+    },
     "logform": {
       "version": "2.2.0",
       "resolved": "https://registry.npmjs.org/logform/-/logform-2.2.0.tgz",
@@ -477,6 +677,27 @@
         "mime-db": "1.44.0"
       }
     },
+    "minimatch": {
+      "version": "3.0.4",
+      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz",
+      "integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==",
+      "requires": {
+        "brace-expansion": "^1.1.7"
+      }
+    },
+    "minimist": {
+      "version": "1.2.5",
+      "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.5.tgz",
+      "integrity": "sha512-FM9nNUYrRBAELZQT3xeZQ7fmMOBg6nWNmJKTcgsJeaLstP/UODVpGsr5OhXhhXg6f+qtJ8uiZ+PUxkDWcgIXLw=="
+    },
+    "mkdirp": {
+      "version": "0.5.5",
+      "resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.5.tgz",
+      "integrity": "sha512-NKmAlESf6jMGym1++R0Ra7wvhV+wFW63FaSOFPwRahvea0gMUcGUhVeAg/0BC0wiv9ih5NYPB1Wn1UEI1/L+xQ==",
+      "requires": {
+        "minimist": "^1.2.5"
+      }
+    },
     "moment": {
       "version": "2.29.1",
       "resolved": "https://registry.npmjs.org/moment/-/moment-2.29.1.tgz",
@@ -487,11 +708,31 @@
       "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
       "integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w=="
     },
+    "multer": {
+      "version": "1.4.2",
+      "resolved": "https://registry.npmjs.org/multer/-/multer-1.4.2.tgz",
+      "integrity": "sha512-xY8pX7V+ybyUpbYMxtjM9KAiD9ixtg5/JkeKUTD6xilfDv0vzzOFcCp4Ljb1UU3tSOM3VTZtKo63OmzOrGi3Cg==",
+      "requires": {
+        "append-field": "^1.0.0",
+        "busboy": "^0.2.11",
+        "concat-stream": "^1.5.2",
+        "mkdirp": "^0.5.1",
+        "object-assign": "^4.1.1",
+        "on-finished": "^2.3.0",
+        "type-is": "^1.6.4",
+        "xtend": "^4.0.0"
+      }
+    },
     "negotiator": {
       "version": "0.6.2",
       "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.2.tgz",
       "integrity": "sha512-hZXc7K2e+PgeI1eDBe/10Ard4ekbfrrqG8Ep+8Jmf4JID2bNg7NvCPOZN+kfF574pFQI7mum2AUqDidoKqcTOw=="
     },
+    "object-assign": {
+      "version": "4.1.1",
+      "resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
+      "integrity": "sha1-IQmtx5ZYh8/AXLvUQsrIv7s2CGM="
+    },
     "object-hash": {
       "version": "2.0.3",
       "resolved": "https://registry.npmjs.org/object-hash/-/object-hash-2.0.3.tgz",
@@ -573,6 +814,11 @@
       "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
       "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ=="
     },
+    "safe-stable-stringify": {
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/safe-stable-stringify/-/safe-stable-stringify-1.1.1.tgz",
+      "integrity": "sha512-ERq4hUjKDbJfE4+XtZLFPCDi8Vb1JqaxAPTxWFLBx8XcAlf9Bda/ZJdVezs/NAfsMQScyIlUMx+Yeu7P7rx5jw=="
+    },
     "safer-buffer": {
       "version": "2.1.2",
       "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
@@ -672,6 +918,11 @@
       "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.5.0.tgz",
       "integrity": "sha1-Fhx9rBd2Wf2YEfQ3cfqZOBR4Yow="
     },
+    "streamsearch": {
+      "version": "0.1.2",
+      "resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-0.1.2.tgz",
+      "integrity": "sha1-gIudDlb8Jz2Am6VzOOkpkZoanxo="
+    },
     "string_decoder": {
       "version": "1.3.0",
       "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
@@ -698,6 +949,14 @@
         "semver": "^7.3.2"
       }
     },
+    "supports-color": {
+      "version": "5.5.0",
+      "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz",
+      "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==",
+      "requires": {
+        "has-flag": "^3.0.0"
+      }
+    },
     "text-hex": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/text-hex/-/text-hex-1.0.0.tgz",
@@ -722,6 +981,11 @@
         "mime-types": "~2.1.24"
       }
     },
+    "typedarray": {
+      "version": "0.0.6",
+      "resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz",
+      "integrity": "sha1-hnrHTjhkGHsdPUfZlqeOxciDB3c="
+    },
     "unpipe": {
       "version": "1.0.0",
       "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
@@ -806,6 +1070,11 @@
           }
         }
       }
+    },
+    "xtend": {
+      "version": "4.0.2",
+      "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+      "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ=="
     }
   }
 }

package.json

@@ -23,8 +23,13 @@
   "homepage": "https://github.com/FoxxMD/maloja-spotify-scrobbler#readme",
   "dependencies": {
     "@awaitjs/express": "^0.6.3",
-    "date-fns": "^2.16.1",
+    "body-parser": "^1.19.0",
+    "dayjs": "^1.9.6",
+    "ejs": "^3.1.5",
     "express": "^4.17.1",
+    "lastfm-node-client": "^2.2.0",
+    "multer": "^1.4.2",
+    "safe-stable-stringify": "^1.1.1",
     "spotify-web-api-node": "^5.0.0",
     "superagent": "^6.1.0",
     "winston": "^3.3.3",

server/middleware.js

@@ -0,0 +1,43 @@
+export const makeSourceCheckMiddle = sources => (req, res, next) => {
+    const {
+        query: {
+            name,
+            type
+        } = {}
+    } = req;
+
+    if (name === undefined) {
+        return res.status(404).send('Source name must be defined');
+    }
+
+    const source = sources.getByNameAndType(name, type);
+
+    if (source === undefined) {
+        return res.status(404).send(`No source with the name [${name}] and type [${type}`);
+    }
+
+    req.sourceName = name;
+    req.scrobbleSource = source;
+    next();
+}
+
+export const makeClientCheckMiddle = clients => (req, res, next) => {
+    const {
+        query: {
+            name
+        } = {}
+    } = req;
+
+    if (name === undefined) {
+        return res.status(404).send('Client name must be defined');
+    }
+
+    const client = clients.getByName(name);
+
+    if (client === undefined) {
+        return res.status(404).send(`No client with the name: ${name}`);
+    }
+
+    req.scrobbleClient = client;
+    next();
+}

sources/AbstractSource.js

@@ -0,0 +1,148 @@
+import dayjs from "dayjs";
+import {buildTrackString, capitalize, createLabelledLogger, sleep} from "../utils.js";
+
+export default class AbstractSource {
+
+    name;
+    type;
+    identifier;
+
+    config;
+    clients;
+    logger;
+
+    canPoll = false;
+    polling = false;
+    tracksDiscovered = 0;
+
+    constructor(type, name, config = {}, clients = []) {
+        this.type = type;
+        this.name = name;
+        this.identifier = `${capitalize(this.type)} - ${name}`;
+        this.logger = createLabelledLogger(this.identifier, this.identifier);
+        this.config = config;
+        this.clients = clients;
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        return [];
+    }
+
+    // by default if the track was recently played it is valid
+    // this is useful for sources where the track doesn't have complete information like Subsonic
+    // TODO make this more descriptive? or move it elsewhere
+    recentlyPlayedTrackIsValid = (playObj) => {
+        return true;
+    }
+
+    poll = async (allClients) => {
+        await this.startPolling(allClients);
+    }
+
+    /**
+     * @param {ScrobbleClients} allClients
+     */
+    startPolling = async (allClients) => {
+        if (this.polling === true) {
+            return;
+        }
+        this.logger.info('Polling started');
+        let lastTrackPlayedAt = dayjs();
+        let checkCount = 0;
+        try {
+            this.polling = true;
+            while (true) {
+                if(this.polling === false) {
+                    this.logger.info('Stopped polling due to user input');
+                    break;
+                }
+                let playObjs = [];
+                this.logger.debug('Refreshing recently played')
+                playObjs = await this.getRecentlyPlayed({formatted: true});
+                checkCount++;
+                let newTracksFound = false;
+                let closeToInterval = false;
+                const now = dayjs();
+
+                const playInfo = playObjs.reduce((acc, playObj) => {
+                    if(this.recentlyPlayedTrackIsValid(playObj)) {
+                        const {data: {playDate} = {}} = playObj;
+                        if (playDate.unix() > lastTrackPlayedAt.unix()) {
+                            newTracksFound = true;
+                            this.logger.info(`New Track => ${buildTrackString(playObj)}`);
+
+                            if (closeToInterval === false) {
+                                closeToInterval = Math.abs(now.unix() - playDate.unix()) < 5;
+                            }
+
+                            return {
+                                plays: [...acc.plays, {...playObj, meta: {...playObj.meta, newFromSource: true}}],
+                                lastTrackPlayedAt: playDate
+                            }
+                        }
+                        return {
+                            ...acc,
+                            plays: [...acc.plays, playObj]
+                        }
+                    }
+                    return acc;
+                }, {plays: [], lastTrackPlayedAt});
+                playObjs = playInfo.plays;
+                lastTrackPlayedAt = playInfo.lastTrackPlayedAt;
+
+                if (closeToInterval) {
+                    // because the interval check was so close to the play date we are going to delay client calls for a few secs
+                    // this way we don't accidentally scrobble ahead of any other clients (we always want to be behind so we can check for dups)
+                    // additionally -- it should be ok to have this in the for loop because played_at will only decrease (be further in the past) so we should only hit this once, hopefully
+                    this.logger.info('Track is close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first');
+                    await sleep(10 * 1000);
+                }
+
+                if (newTracksFound === false) {
+                    if (playObjs.length === 0) {
+                        this.logger.debug(`No new tracks found and no tracks returned from API`);
+                    } else {
+                        this.logger.debug(`No new tracks found. Newest track returned was ${buildTrackString(playObjs.slice(-1)[0])}`);
+                    }
+                } else {
+                    checkCount = 0;
+                }
+
+                const scrobbleResult = await allClients.scrobble(playObjs, {
+                    forceRefresh: closeToInterval,
+                    scrobbleFrom: this.identifier,
+                    scrobbleTo: this.clients
+                });
+
+                if (scrobbleResult.length > 0) {
+                    checkCount = 0;
+                    this.tracksDiscovered += scrobbleResult.length;
+                }
+
+                const {interval = 30} = this.config;
+
+                let sleepTime = interval;
+                // don't need to do back off calc if interval is 10 minutes or greater since its already pretty light on API calls
+                // and don't want to back off if we just started the app
+                if (checkCount > 5 && sleepTime < 600) {
+                    const lastPlayToNowSecs = Math.abs(now.unix() - lastTrackPlayedAt.unix());
+                    // back off if last play was longer than 10 minutes ago
+                    const backoffThreshold = Math.min((interval * 10), 600);
+                    if (lastPlayToNowSecs >= backoffThreshold) {
+                        // back off to a maximum of 5 minutes
+                        sleepTime = Math.min(interval * 5, 300);
+                    }
+                }
+
+                // sleep for interval
+                this.logger.debug(`Sleeping for ${sleepTime}s`);
+                await sleep(sleepTime * 1000);
+
+            }
+        } catch (e) {
+            this.logger.error('Error occurred while polling');
+            this.logger.error(e);
+            this.polling = false;
+        }
+    }
+}

sources/PlexSource.js

@@ -0,0 +1,173 @@
+import dayjs from "dayjs";
+import {buildTrackString} from "../utils.js";
+import AbstractSource from "./AbstractSource.js";
+
+export default class PlexSource extends AbstractSource {
+    users;
+    libraries;
+    servers;
+
+    constructor(name, config, clients, type = 'plex') {
+        super(type, name, config, clients);
+        const {user, libraries, servers} = config
+
+        if (user === undefined || user === null) {
+            this.users = undefined;
+        } else {
+            if (!Array.isArray(user)) {
+                this.users = [user];
+            } else {
+                this.users = user;
+            }
+            this.users = this.users.map(x => x.toLocaleLowerCase())
+        }
+
+        if (libraries === undefined || libraries === null) {
+            this.libraries = undefined;
+        } else {
+            if (!Array.isArray(libraries)) {
+                this.libraries = [libraries];
+            } else {
+                this.libraries = libraries;
+            }
+            this.libraries = this.libraries.map(x => x.toLocaleLowerCase())
+        }
+
+        if (servers === undefined || servers === null) {
+            this.servers = undefined;
+        } else {
+            if (!Array.isArray(servers)) {
+                this.servers = [servers];
+            } else {
+                this.servers = servers;
+            }
+            this.servers = this.servers.map(x => x.toLocaleLowerCase())
+        }
+
+        if (user === undefined && libraries === undefined && servers === undefined) {
+            this.logger.warn('Initializing, but with no filters! All tracks from all users on all servers and libraries will be scrobbled.');
+        } else {
+            this.logger.info(`Initializing with the following filters => Users: ${this.users === undefined ? 'N/A' : this.users.join(', ')} | Libraries: ${this.libraries === undefined ? 'N/A' : this.libraries.join(', ')} | Servers: ${this.servers === undefined ? 'N/A' : this.servers.join(', ')}`);
+        }
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            event,
+            Account: {
+                title: user,
+            } = {},
+            Metadata: {
+                type,
+                title: track,
+                parentTitle: album,
+                grandparentTitle: artist,
+                librarySectionTitle: library
+            } = {},
+            Server: {
+                title: server
+            } = {},
+        } = obj;
+        return {
+            data: {
+                artists: [artist],
+                album,
+                track,
+                playDate: dayjs(),
+            },
+            meta: {
+                event,
+                mediaType: type,
+                user,
+                library,
+                server,
+                source: 'Plex',
+                newFromSource,
+            }
+        }
+    }
+
+    isValidEvent = (playObj) => {
+        const {
+            meta: {
+                mediaType, event, user, library, server
+            },
+            data: {
+                artists,
+                track,
+            } = {}
+        } = playObj;
+
+        const hint = this.type === 'tautulli' ? ' (Check notification agent json data configuration)' : '';
+
+        if (this.users !== undefined) {
+            if (user === undefined) {
+                this.logger.warn(`Config defined users but payload contained no user info${hint}`);
+            } else if (!this.users.includes(user.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because author was not an allowed user: ${user}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        if (event !== undefined && event !== 'media.scrobble') {
+            this.logger.debug(`Will not scrobble event because it is not media.scrobble (${event})`, {
+                artists,
+                track
+            })
+            return false;
+        }
+
+        if (mediaType !== 'track') {
+            this.logger.debug(`Will not scrobble event because media type was not a track (${mediaType})`, {
+                artists,
+                track
+            });
+            return false;
+        }
+
+        if (this.libraries !== undefined) {
+            if (library === undefined) {
+                this.logger.warn(`Config defined libraries but payload contained no library info${hint}`);
+            } else if (!this.libraries.includes(library.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because library was not on allowed list: ${library}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        if (this.servers !== undefined) {
+            if (server === undefined) {
+                this.logger.warn(`Config defined server but payload contained no server info${hint}`);
+            } else if (!this.servers.includes(server.toLocaleLowerCase())) {
+                this.logger.debug(`Will not scrobble event because server was not on allowed list: ${server}`, {
+                    artists,
+                    track
+                })
+                return false;
+            }
+        }
+
+        return true;
+    }
+
+    handle = async (playObj, allClients) => {
+        if (!this.isValidEvent(playObj)) {
+            return;
+        }
+
+        this.logger.info(`New Track => ${buildTrackString(playObj)}`);
+        try {
+            await allClients.scrobble(playObj, {scrobbleTo: this.clients, scrobbleFrom: this.identifier});
+            // only gets hit if we scrobbled ok
+            this.tracksDiscovered++;
+        } catch (e) {
+            this.logger.error('Encountered error while scrobbling')
+            this.logger.error(e)
+        }
+    }
+}

sources/ScrobbleSources.js

@@ -0,0 +1,240 @@
+import {createLabelledLogger, isValidConfigStructure, readJson} from "../utils.js";
+import SpotifySource from "./SpotifySource.js";
+import PlexSource from "./PlexSource.js";
+import TautulliSource from "./TautulliSource.js";
+import {SubsonicSource} from "./SubsonicSource.js";
+
+export default class ScrobbleSources {
+
+    sources = [];
+    logger;
+    configDir;
+    localUrl;
+
+    sourceTypes = ['spotify', 'plex', 'tautulli', 'subsonic'];
+
+    constructor(localUrl, configDir = process.cwd()) {
+        this.configDir = configDir;
+        this.localUrl = localUrl;
+        this.logger = createLabelledLogger('sources', 'Sources');
+    }
+
+    getByName = (name) => {
+        return this.sources.find(x => x.name === name);
+    }
+
+    getByType = (type) => {
+        return this.sources.filter(x => x.type === type);
+    }
+
+    getByNameAndType = (name, type) => {
+        return this.sources.find(x => x.name === name && x.type === type);
+    }
+
+    buildSourcesFromConfig = async (additionalConfigs = []) => {
+        let configs = additionalConfigs;
+
+        let configFile;
+        try {
+            configFile = await readJson(`${this.configDir}/config.json`, {throwOnNotFound: false});
+        } catch (e) {
+            throw new Error('config.json could not be parsed');
+        }
+        if (configFile !== undefined) {
+            const {sources: mainConfigSourcesConfigs = []} = configFile;
+            if (!mainConfigSourcesConfigs.every(x => x !== null && typeof x === 'object')) {
+                throw new Error('All sources from config.json must be objects');
+            }
+            for (const c of mainConfigSourcesConfigs) {
+                const {name = 'unnamed'} = c;
+                configs.push({...c, name, source: 'config.json'});
+            }
+        }
+
+        for (let sourceType of this.sourceTypes) {
+            // env builder for single user mode
+            switch (sourceType) {
+                case 'spotify':
+                    const s = {
+                        accessToken: process.env.SPOTIFY_ACCESS_TOKEN,
+                        clientId: process.env.SPOTIFY_CLIENT_ID,
+                        clientSecret: process.env.SPOTIFY_CLIENT_SECRET,
+                        redirectUri: process.env.SPOTIFY_REDIRECT_URI,
+                        refreshToken: process.env.SPOTIFY_REFRESH_TOKEN,
+                    };
+                    if (!Object.values(s).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'spotify',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: s
+                        })
+                    }
+                    break;
+                case 'tautulli':
+                    const t = {
+                        // support this for now
+                        user: process.env.TAUTULLI_USER || process.env.PLEX_USER
+                    };
+                    if (!Object.values(t).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'tautulli',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: t
+                        })
+                    }
+                    break;
+                case 'plex':
+                    const p = {
+                        user: process.env.PLEX_USER
+                    };
+                    if (!Object.values(p).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'plex',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: p
+                        })
+                    }
+                    break;
+                case 'subsonic':
+                    const sub = {
+                        user: process.env.SUBSONIC_USER,
+                        password: process.env.SUBSONIC_PASSWORD,
+                        url: process.env.SUBSONIC_URL,
+                    };
+                    if (!Object.values(sub).every(x => x === undefined)) {
+                        configs.push({
+                            type: 'subsonic',
+                            name: 'unnamed',
+                            source: 'ENV',
+                            mode: 'single',
+                            data: sub
+                        })
+                    }
+                    break;
+                default:
+                    break;
+            }
+            let rawSourceConfigs;
+            try {
+                rawSourceConfigs = await readJson(`${this.configDir}/${sourceType}.json`, {throwOnNotFound: false});
+            } catch (e) {
+                throw new Error(`${sourceType}.json config file could not be parsed`);
+            }
+            if (rawSourceConfigs !== undefined) {
+                let sourceConfigs = [];
+                if (Array.isArray(rawSourceConfigs)) {
+                    sourceConfigs = rawSourceConfigs;
+                } else if (rawSourceConfigs === null || typeof rawSourceConfigs === 'object') {
+                    // backwards compatibility, assuming its single-user mode
+                    if (rawSourceConfigs.data === undefined) {
+                        sourceConfigs = [{data: rawSourceConfigs, mode: 'single', name: 'unnamed'}];
+                    } else {
+                        sourceConfigs = [rawSourceConfigs];
+                    }
+                } else {
+                    throw new Error(`All top level data from ${sourceType}.json must be an object or array of objects`);
+                }
+                for (const m of sourceConfigs) {
+                    if (m === null || typeof m !== 'object') {
+                        throw new Error(`All top-level data from ${sourceType}.json must be an object or array of objects`);
+                    }
+                    m.source = `${sourceType}.json`;
+                    m.type = sourceType;
+                    configs.push(m);
+                }
+            }
+        }
+
+        // we have all possible configurations so we'll check they are minimally valid
+        const configErrors = configs.reduce((acc, c) => {
+            const isValid = isValidConfigStructure(c, {type: true, data: true});
+            if (isValid !== true) {
+                const msg = `Source config from ${c.source} with name [${c.name || 'unnamed'}] of type [${c.type || 'unknown'}] has errors: ${isValid.join(' | ')}`;
+                return acc.concat(msg);
+            }
+            return acc;
+        }, []);
+        if (configErrors.length > 0) {
+            for (const m of configErrors) {
+                this.logger.error(m);
+            }
+            throw new Error('Could not build sources due to above errors');
+        }
+
+        // finally! all configs are valid, structurally, and can now be passed to addClient
+        // do a last check that names (within each type) are unique and warn if not, but add anyways
+        const typeGroupedConfigs = configs.reduce((acc, curr) => {
+            const {type} = curr;
+            const {[type]: t = []} = acc;
+            return {...acc, [type]: [...t, curr]};
+        }, {});
+        // only need to warn if dup names PER TYPE
+        for (const [type, typedConfigs] of Object.entries(typeGroupedConfigs)) {
+            const nameGroupedConfigs = typedConfigs.reduce((acc, curr) => {
+                const {name = 'unnamed'} = curr;
+                const {[name]: n = []} = acc;
+                return {...acc, [name]: [...n, curr]};
+            }, {});
+            for (const [name, namedConfigs] of Object.entries(nameGroupedConfigs)) {
+                let tempNamedConfigs = namedConfigs;
+                const hasDups = namedConfigs.length > 1;
+                if (hasDups) {
+                    const sources = namedConfigs.map(c => `Config object from ${c.source} of type [${c.type}]`);
+                    this.logger.warn(`Source configs have naming conflicts -- the following configs have the same name "${name}":\n\n${sources.join('\n')}\n`);
+                    if (name === 'unnamed') {
+                        this.logger.info('HINT: "unnamed" configs occur when using ENVs, if a multi-user mode config does not have a "name" property, or if a config is built in single-user mode');
+                    }
+                }
+                tempNamedConfigs = tempNamedConfigs.map(({name = 'unnamed', ...x}, i) => ({
+                    ...x,
+                    name: hasDups ? `${name}${i + 1}` : name
+                }));
+                for (const c of tempNamedConfigs) {
+                    await this.addSource(c);
+                }
+            }
+        }
+    }
+
+    addSource = async (clientConfig) => {
+        const isValidConfig = isValidConfigStructure(clientConfig, {name: true, data: true, type: true});
+        if (isValidConfig !== true) {
+            throw new Error(`Config object from ${clientConfig.source || 'unknown'} with name [${clientConfig.name || 'unnamed'}] of type [${clientConfig.type || 'unknown'}] has errors: ${isValidConfig.join(' | ')}`)
+        }
+        const {type, name, clients = [], data = {}} = clientConfig;
+        this.logger.debug(`(${name}) Initializing ${type} source`);
+        switch (type) {
+            case 'spotify':
+                const spotifySource = new SpotifySource(name, {
+                    ...data,
+                    localUrl: this.localUrl,
+                    configDir: this.configDir
+                }, clients);
+                await spotifySource.buildSpotifyApi();
+                this.sources.push(spotifySource);
+                break;
+            case 'plex':
+                const plexSource = await new PlexSource(name, data, clients);
+                this.sources.push(plexSource);
+                break;
+            case 'tautulli':
+                const tautulliSource = await new TautulliSource(name, data, clients);
+                this.sources.push(tautulliSource);
+                break;
+            case 'subsonic':
+                const ssSource = new SubsonicSource(name, data, clients);
+                await ssSource.testConnection();
+                this.sources.push(ssSource);
+                break;
+            default:
+                break;
+        }
+        this.logger.info(`(${name}) ${type} source initialized`);
+    }
+}

sources/SpotifySource.js

@@ -0,0 +1,223 @@
+import dayjs from "dayjs";
+import {
+    readJson,
+    writeFile,
+    sortByPlayDate,
+} from "../utils.js";
+import SpotifyWebApi from "spotify-web-api-node";
+import AbstractSource from "./AbstractSource.js";
+
+const scopes = ['user-read-recently-played', 'user-read-currently-playing'];
+const state = 'random';
+
+export default class SpotifySource extends AbstractSource {
+
+    spotifyApi;
+    localUrl;
+    workingCredsPath;
+    configDir;
+
+    constructor(name, config = {}, clients = []) {
+        super('spotify', name, config, clients);
+        const {
+            localUrl,
+            configDir,
+            interval = 60,
+        } = config;
+
+        if (interval < 15) {
+            this.logger.warn('Interval should be above 30 seconds...😬');
+        }
+
+        this.config.interval = interval;
+
+        this.configDir = configDir;
+        this.workingCredsPath = `${configDir}/currentCreds-${name}.json`;
+        this.localUrl = localUrl;
+        this.canPoll = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            track: {
+                artists = [],
+                name,
+                id,
+                duration_ms,
+                album: {
+                    name: albumName,
+                } = {},
+                external_urls: {
+                    spotify,
+                } = {}
+            } = {},
+            played_at
+        } = obj;
+        //let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(',');
+        return {
+            data: {
+                artists: artists.map(x => x.name),
+                album: albumName,
+                track: name,
+                duration: duration_ms / 1000,
+                playDate: dayjs(played_at),
+            },
+            meta: {
+                trackLength: duration_ms / 1000,
+                source: 'Spotify',
+                sourceId: id,
+                newFromSource,
+                url: {
+                    web: spotify
+                }
+            }
+        }
+    }
+
+    buildSpotifyApi = async () => {
+
+        let spotifyCreds = {};
+        try {
+            spotifyCreds = await readJson(this.workingCredsPath, {throwOnNotFound: false});
+        } catch (e) {
+            this.logger.warn('Current spotify credentials file exists but could not be parsed', { path: this.workingCredsPath });
+        }
+
+        const {
+            accessToken,
+            clientId,
+            clientSecret,
+            redirectUri,
+            refreshToken,
+        } = this.config || {};
+
+        const rdUri = redirectUri || `${this.localUrl}/callback`;
+
+
+        const {token = accessToken, refreshToken: rt = refreshToken} = spotifyCreds || {};
+
+        const apiConfig = {
+            clientId,
+            clientSecret,
+            accessToken: token,
+            refreshToken: rt,
+        }
+
+        if (Object.values(apiConfig).every(x => x === undefined)) {
+            this.logger.info('No values found for Spotify configuration, skipping initialization');
+            return;
+        }
+        apiConfig.redirectUri = rdUri;
+
+        const validationErrors = [];
+
+        if (token === undefined) {
+            if (clientId === undefined) {
+                validationErrors.push('clientId must be defined when access token is not present');
+            }
+            if (clientSecret === undefined) {
+                validationErrors.push('clientSecret must be defined when access token is not present');
+            }
+            if (rdUri === undefined) {
+                validationErrors.push('redirectUri must be defined when access token is not present');
+            }
+            if (validationErrors.length !== 0) {
+                validationErrors.unshift('no access token is defined');
+            }
+        } else if (rt === undefined && (
+            clientId === undefined ||
+            clientSecret === undefined ||
+            rdUri === undefined
+        )) {
+            this.logger.warn('Access token is present but no refresh token is defined and remaining configuration is not sufficient to re-authorize. Without a refresh token API calls will fail after current token is expired.');
+        }
+
+        if (validationErrors.length !== 0) {
+            this.logger.warn(`Configuration was not valid:\*${validationErrors.join('\n')}`);
+            throw new Error('Failed to initialize a Spotify source');
+        }
+
+        this.spotifyApi = new SpotifyWebApi(apiConfig);
+    }
+
+    createAuthUrl = () => {
+        return this.spotifyApi.createAuthorizeURL(scopes, this.name);
+    }
+
+    handleAuthCodeCallback = async ({error, code}) => {
+        if (error === undefined) {
+            const tokenResponse = await this.spotifyApi.authorizationCodeGrant(code);
+            this.spotifyApi.setAccessToken(tokenResponse.body['access_token']);
+            this.spotifyApi.setRefreshToken(tokenResponse.body['refresh_token']);
+            await writeFile(this.workingCredsPath, JSON.stringify({
+                token: tokenResponse.body['access_token'],
+                refreshToken: tokenResponse.body['refresh_token']
+            }));
+            this.logger.info('Got token from code grant authorization!');
+            return true;
+        } else {
+            this.logger.warn('Callback contained an error! User may have denied access?')
+            this.logger.error(error);
+            return error;
+        }
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        const {limit = 20, formatted = false} = options;
+        const func = api => api.getMyRecentlyPlayedTracks({
+            limit
+        });
+        const result = await this.callApi(func);
+        if (formatted === true) {
+            return result.body.items.map(x => SpotifySource.formatPlayObj(x)).sort(sortByPlayDate);
+        }
+        return result;
+    }
+
+    callApi = async (func) => {
+        try {
+            return await func(this.spotifyApi);
+        } catch (e) {
+            if (e.statusCode === 401) {
+                if (this.spotifyApi.getRefreshToken() === undefined) {
+                    throw new Error('Access token was not valid and no refresh token was present')
+                }
+                this.logger.debug('Access token was not valid, attempting to refresh');
+
+                const tokenResponse = await this.spotifyApi.refreshAccessToken();
+                const {
+                    body: {
+                        access_token,
+                        // spotify may return a new refresh token
+                        // if it doesn't then continue to use the last refresh token we received
+                        refresh_token = this.spotifyApi.getRefreshToken(),
+                    } = {}
+                } = tokenResponse;
+                this.spotifyApi.setAccessToken(access_token);
+                await writeFile(this.workingCredsPath, JSON.stringify({
+                    token: access_token,
+                    refreshToken: refresh_token,
+                }));
+                try {
+                    return await func(this.spotifyApi);
+                } catch (ee) {
+                    this.logger.error('Refreshing access token encountered an error');
+                    this.logger.error(ee, {label: 'Spotify'});
+                    throw ee;
+                }
+            } else {
+                this.logger.error('Refreshing access token encountered an error');
+                this.logger.error(e, {label: 'Spotify'});
+                throw e;
+            }
+        }
+    }
+
+    poll = async (allClients) => {
+        if (this.spotifyApi === undefined) {
+            this.logger.warn('Cannot poll spotify without valid credentials configuration')
+            return;
+        }
+        await this.startPolling(allClients);
+    }
+}

sources/SubsonicSource.js

@@ -0,0 +1,144 @@
+import AbstractSource from "./AbstractSource.js";
+import request from 'superagent';
+import crypto from 'crypto';
+import dayjs from "dayjs";
+import isSameOrAfter from "dayjs/plugin/isSameOrAfter.js";
+import {buildTrackString} from "../utils.js";
+
+dayjs.extend(isSameOrAfter);
+
+export class SubsonicSource extends AbstractSource {
+
+    constructor(name, config = {}, clients = []) {
+        super('subsonic', name, config, clients);
+
+        const {user, password, url} = this.config;
+
+        if (user === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'user' is not defined`);
+        }
+        if (password === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'password' is not defined`);
+        }
+        if (url === undefined) {
+            throw new Error(`Cannot setup Subsonic source, 'url' is not defined`);
+        }
+
+        this.canPoll = true;
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            id,
+            title,
+            album,
+            artist,
+            duration, // seconds
+            minutesAgo,
+        } = obj;
+        return {
+            data: {
+                artists: [artist],
+                album,
+                track: title,
+                duration,
+                // subsonic doesn't return an exact datetime, only how many whole minutes ago it was played
+                // so we need to force the time to be 0 seconds always so that when we compare against scrobbles from client the time isn't off
+                playDate: minutesAgo === 0 ? dayjs().startOf('minute') : dayjs().startOf('minute').subtract(minutesAgo, 'minute'),
+            },
+            meta: {
+                trackLength: duration,
+                source: 'Subsonic',
+                sourceId: id,
+                newFromSource,
+            }
+        }
+    }
+
+    recentlyPlayedTrackIsValid = (playObj) => {
+        const {data: {playDate} = {}} = playObj;
+        // want to make sure that the track has been played for at least one minute (according to subsonic api)
+        const isValid = dayjs().startOf('minute').subtract(1, 'minute').isSameOrAfter(playDate);
+        if (!isValid) {
+            this.logger.debug(`${buildTrackString(playObj, {include: ['artist', 'track']})} recently played but not valid b/c it has not been playing for >= 1 minute`);
+        }
+        return isValid;
+    }
+
+    callApi = async (req) => {
+        const {user, password} = this.config;
+
+        const salt = await crypto.randomBytes(10).toString('hex');
+        const hash = crypto.createHash('md5').update(`${password}${salt}`).digest('hex')
+        req.query({
+            u: user,
+            t: hash,
+            s: salt,
+            v: '1.15.0',
+            c: `multi-scrobbler - ${this.name}`,
+            f: 'json'
+        })
+        try {
+            const resp = await req;
+            const {
+                body: {
+                    "subsonic-response": {
+                        status,
+                    },
+                    "subsonic-response": ssResp = {}
+                } = {}
+            } = resp;
+            if (status === 'failed') {
+                const err = new Error('Subsonic API returned an error');
+                err.response = resp;
+                throw  err;
+            }
+            return ssResp;
+        } catch (e) {
+            const {
+                message,
+                response: {
+                    status,
+                    body: {
+                        "subsonic-response": {
+                            status: ssStatus,
+                            error: {
+                                code,
+                                message: ssMessage,
+                            } = {},
+                        } = {},
+                        "subsonic-response": ssResp
+                    } = {},
+                    text,
+                } = {},
+                response,
+            } = e;
+            let msg = response !== undefined ? `API Call failed: Server Response => ${ssMessage}` : `API Call failed: ${message}`;
+            const responseMeta = ssResp ?? text;
+            this.logger.error(msg, {status, response: responseMeta});
+            throw e;
+        }
+    }
+
+    testConnection = async () => {
+        const {url} = this.config;
+        try {
+            await this.callApi(request.get(`${url}/rest/ping`));
+            this.logger.info('Subsonic API Status: ok');
+        } catch (e) {
+            this.logger.error(e);
+        }
+    }
+
+    getRecentlyPlayed = async (options = {}) => {
+        const {formatted = false} = options;
+        const {url} = this.config;
+        const resp = await this.callApi(request.get(`${url}/rest/getNowPlaying`));
+        const {
+            nowPlaying: {
+                entry = []
+            } = {}
+        } = resp;
+        return entry.map(x => formatted ? SubsonicSource.formatPlayObj(x) : x)
+    }
+}

sources/TautulliSource.js

@@ -0,0 +1,47 @@
+import dayjs from "dayjs";
+import PlexSource from "./PlexSource.js";
+
+export default class TautulliSource extends PlexSource {
+
+    constructor(name, config, clients) {
+        super(name, config, clients, 'tautulli');
+    }
+
+    static formatPlayObj(obj, newFromSource = false) {
+        const {
+            artist_name,
+            track_name,
+            track_artist,
+            album_name,
+            media_type,
+            title,
+            library_name,
+            server,
+            duration,
+            username,
+        } = obj;
+        let artists = [artist_name];
+        if (track_artist !== undefined && track_artist !== artist_name) {
+            artists.push(track_artist);
+        }
+        return {
+            data: {
+                artists,
+                album: album_name,
+                track: track_name,
+                duration,
+                playDate: dayjs(),
+            },
+            meta: {
+                title,
+                library: library_name,
+                server,
+                mediaType: media_type,
+                user: username,
+                trackLength: duration,
+                source: 'Tautulli',
+                newFromSource,
+            }
+        }
+    }
+}

utils.js

@@ -1,18 +1,36 @@
 import fs, {promises, constants} from "fs";
+import dayjs from 'dayjs';
+import utc from 'dayjs/plugin/utc.js';
+import winston from "winston";
+import jsonStringify from 'safe-stable-stringify';
 
-export async function readJson(path) {
-    await promises.access(path, constants.R_OK);
-    const data = await promises.readFile(path);
-    return JSON.parse(data);
+const {format} = winston;
+const {combine, printf, timestamp, label, splat, errors} = format;
 
-    // return new Promise((resolve, reject) => {
-    //     fs.readFile(path, 'utf8', function (err, data) {
-    //         if (err) {
-    //             reject(err);
-    //         }
-    //         resolve(JSON.parse(data));
-    //     });
-    // });
+dayjs.extend(utc);
+
+export async function readJson(path, {logErrors = true, throwOnNotFound = true} = {}) {
+    try {
+        await promises.access(path, constants.R_OK);
+        const data = await promises.readFile(path);
+        return JSON.parse(data);
+    } catch (e) {
+        const {code} = e;
+        if (code === 'ENOENT') {
+            if (throwOnNotFound) {
+                if (logErrors) {
+                    this.logger.warn('No file found at given path', {filePath: path});
+                }
+                throw e;
+            } else {
+                return;
+            }
+        } else if (logErrors) {
+            this.logger.warn(`Encountered error while parsing file`, {filePath: path});
+            this.logger.error(e);
+        }
+        throw e;
+    }
 }
 
 export async function readText(path) {
@@ -49,18 +67,196 @@ export function sleep(ms) {
     return new Promise(resolve => setTimeout(resolve, ms));
 }
 
-export const buildTrackString = (obj) => {
-    const {
-        track: {
-            artists = [],
-            name,
-            id,
-            external_urls: {
-                spotify,
-            } = {}
-        } = {},
-        played_at
-    } = obj;
-    let artistString = artists.reduce((acc, curr) => acc.concat(curr.name), []).join(' / ');
-    return `${artistString} - ${name}, played at ${played_at}`
+export const longestString = strings => strings.reduce((acc, curr) => curr.length > acc ? curr.length : acc, 0);
+export const truncateStringArrToLength = (length, truncStr = '...') => {
+    const truncater = truncateStringToLength(length, truncStr);
+    return strings => strings.map(truncater);
+}
+export const truncateStringToLength = (length, truncStr = '...') => str => str.length > length ? `${str.slice(0, length)}${truncStr}` : str;
+
+const defaultTransformer = input => input;
+
+export const buildTrackString = (playObj, options = {}) => {
+    const {
+        include = ['time', 'artist', 'track'],
+        transformers: {
+            artists: artistsFunc = a => a.join(' / '),
+            track: trackFunc = defaultTransformer,
+            time: timeFunc = t => t.local().format(),
+            timeFromNow = t => t.local().fromNow(),
+        } = {}
+    } = options;
+    const {
+        data: {
+            artists,
+            album,
+            track,
+            playDate
+        } = {}
+    } = playObj;
+
+    let str = '';
+    if(include.includes('artist')) {
+        str = `${artistsFunc(artists)}`;
+    }
+    if(include.includes('track')) {
+        if(str !== '') {
+            str = `${str} - ${trackFunc(track)}`;
+        } else {
+            str = `${trackFunc(track)}`;
+        }
+    }
+    if (include.includes('time')) {
+        str = `${str} @ ${timeFunc(playDate)}`
+    }
+    if (include.includes('timeFromNow')) {
+        str = `${str} (${timeFromNow(playDate)})`
+    }
+    return str;
+}
+
+// sorts playObj formatted objects by playDate in ascending (oldest first) order
+export const sortByPlayDate = (a, b) => a.data.playDate.isAfter(b.data.playDate) ? 1 : -1;
+
+const s = splat();
+const SPLAT = Symbol.for('splat')
+const errorsFormat = errors({stack: true});
+const CWD = process.cwd();
+
+let longestLabel = 3;
+export const defaultFormat = printf(({level, message, label = 'App', timestamp, [SPLAT]: splatObj, stack, ...rest}) => {
+    let stringifyValue = splatObj !== undefined ? jsonStringify(splatObj) : '';
+    if (label.length > longestLabel) {
+        longestLabel = label.length;
+    }
+    let msg = message;
+    let stackMsg = '';
+    if (stack !== undefined) {
+        const stackArr = stack.split('\n');
+        msg = stackArr[0];
+        const cleanedStack = stackArr
+            .slice(1) // don't need actual error message since we are showing it as msg
+            .map(x => x.replace(CWD, 'CWD')) // replace file location up to cwd for user privacy
+            .join('\n'); // rejoin with newline to preserve formatting
+        stackMsg = `\n${cleanedStack}`;
+    }
+
+    return `${timestamp} ${level.padEnd(7)}: [${label.padEnd(longestLabel)}] ${msg}${stringifyValue !== '' ? ` ${stringifyValue}` : ''}${stackMsg}`;
+});
+
+export const labelledFormat = (labelName = 'App') => {
+    const l = label({label: labelName, message: false});
+    return combine(
+        timestamp(
+            {
+                format: () => dayjs().local().format(),
+            }
+        ),
+        l,
+        s,
+        errorsFormat,
+        defaultFormat,
+    );
+}
+
+export const createLabelledLogger = (name = 'default', label = 'App') => {
+    if (winston.loggers.has(name)) {
+        return winston.loggers.get(name);
+    }
+    const def = winston.loggers.get('default');
+    winston.loggers.add(name, {
+        transports: def.transports,
+        level: def.level,
+        format: labelledFormat(label)
+    });
+    return winston.loggers.get(name);
+}
+
+export const setIntersection = (setA, setB) => {
+    let _intersection = new Set()
+    for (let elem of setB) {
+        if (setA.has(elem)) {
+            _intersection.add(elem)
+        }
+    }
+    return _intersection
+}
+
+export const isValidConfigStructure = (obj, required = {}) => {
+    const {name = false, type = false, data = true} = required;
+    const errs = [];
+    if (obj.type === undefined && type) {
+        errs.push("'type' must be defined");
+    }
+    if (obj.name === undefined && name) {
+        errs.push("'name' must be defined")
+    }
+    if (obj.data === undefined && data) {
+        errs.push("'data' must be defined");
+    }
+    if (errs.length > 0) {
+        return errs;
+    }
+    return true;
+}
+
+export const returnDuplicateStrings = (arr) => {
+    const alreadySeen = [];
+    const dupes = [];
+
+    arr.forEach(str => alreadySeen[str] ? dupes.push(str) : alreadySeen[str] = true);
+    return dupes;
+}
+
+export const capitalize = (str) => {
+    return str.charAt(0).toUpperCase() + str.slice(1)
+}
+
+export const playObjDataMatch = (a, b) => {
+    const {
+        data: {
+            artists: aArtists = [],
+            album: aAlbum,
+            track: aTrack,
+        } = {},
+        meta: {
+            source: aSource,
+            sourceId: aSourceId,
+        } = {},
+    } = a;
+
+    const {
+        data: {
+            artists: bArtists = [],
+            album: bAlbum,
+            track: bTrack,
+        } = {},
+        meta: {
+            source: bSource,
+            sourceId: bSourceId,
+        } = {},
+    } = b;
+
+    // if sources are the same and both plays have source ids then we can just compare by id
+    if(aSource === bSource && aSourceId !== undefined && bSourceId !== undefined) {
+        if(aSourceId !== bSourceId) {
+            return false;
+        }
+    }
+
+    if (aTrack !== bTrack) {
+        return false;
+    }
+    if (aAlbum !== bAlbum) {
+        return false;
+    }
+    if (aArtists.length !== bArtists.length) {
+        return false;
+    }
+    // check if every artist from either playObj matches (one way or another) with the artists from the other play obj
+    if (!aArtists.every(x => bArtists.includes(x)) && bArtists.every(x => aArtists.includes(x))) {
+        return false
+    }
+
+    return true;
 }

views/recent.ejs

@@ -0,0 +1,26 @@
+<html>
+<head>
+<style>
+    small {
+        display: block;
+    }
+    .bold {
+        font-weight: bold;
+    }
+    .capitalize {
+        text-transform: capitalize;
+    }
+</style>
+</head>
+<body>
+    <h2>Recently Played (<%= name %>)</h2>
+    <% if (sourceType === 'spotify') { %>
+        This list was returned from the Spotify Api <b>now.</b> Sometimes Spotify "backlogs" user activity so your recent tracks may not show up for minutes (or hours). ¯\_(ツ)_/¯
+    <% } %>
+    <ul>
+        <% plays.forEach(function (play){ %>
+            <li><pre><%-play%></pre></li>
+        <% }) %>
+    </ul>
+</body>
+</html>

views/status.ejs

@@ -0,0 +1,75 @@
+<html>
+<head>
+<style>
+    small {
+        display: block;
+    }
+    .bold {
+        font-weight: bold;
+    }
+    .capitalize {
+        text-transform: capitalize;
+    }
+    .facetContainer {
+        flex-wrap: wrap;
+        display: flex;
+    }
+    .facetItem {
+        flex: 1 0 auto;
+    }
+    .disabled {
+        color: currentColor;
+        cursor: not-allowed;
+        opacity: 0.5;
+        text-decoration: none;
+    }
+</style>
+</head>
+<body>
+<h2>Sources</h2>
+<div class="facetContainer">
+    <% sources.forEach(function (source){ %>
+    <div class="facetItem">
+        <h3><%= source.display %> - <%= source.name %></h3>
+        <ul>
+            <li><b>Status: <%= source.status %></b></li>
+            <li>Tracks Discovered (since app started): <%= source.tracksDiscovered %></li>
+            <% if (source.canPoll === true) { %>
+            <li>Click <a target="_blank" <% if (!source.hasAuth || source.authed) { %> href="recent?name=<%= source.name %>&type=<%= source.type %>" <% } else { %>class="disabled"<% } %>>to see recently played tracks returned by API</a></li>
+                <% if (source.hasAuth) { %>
+            <li>Click <a target="_blank" href="source/auth?name=<%= source.name %>&type=<%= source.type %>">to (re)authenticate and (re)start polling</a></li>
+                <% } %>
+            <li>Click <a target="_blank" <% if (!source.hasAuth || source.authed) { %> href="poll?name=<%= source.name %>&type=<%= source.type %>" <% } else { %>class="disabled"<% } %>>to restart polling</a></li>
+            <% } %>
+        </ul>
+    </div>
+    <% }) %>
+</div>
+<h2>Clients</h2>
+<div class="facetContainer">
+    <% clients.forEach(function (client){ %>
+        <div class="facetItem">
+            <h3><%= client.display %> - <%= client.name %></h3>
+            <ul>
+                <li><b>Status: <%= client.status %></b></li>
+                <li>Tracks Scrobbled (since app started): <%= client.tracksDiscovered %></li>
+                <% if (client.hasAuth) { %>
+                    <li>Click <a target="_blank" href="client/auth?name=<%= client.name %>&type=<%= client.type %>">to (re)authenticate or initialize</a></li>
+                <% } %>
+            </ul>
+        </div>
+    <% }) %>
+</div>
+
+    <h2>Log (Most Recent)</h2>
+    <ul>
+        <li>Level : <%- logs.level %> </li>
+        <li>Sort  : <%- logs.sort %></li>
+        <li>Limit : <%- logs.limit %></li>
+    </ul>
+<pre>
+<% logs.output.forEach(function (logEntry){ %>
+<%=logEntry%><% }) %>
+</pre>
+</body>
+</html>