feat: add observability, debugging, and diagnostics across all layers

Phase 1 — Server-side logging:
- Enhance HTTP request logging with client_ip, bytes, req_id
- Enrich WS disconnect logs with duration, msgs received/sent/dropped,
  voice channel, and last error
- Add structured logging to LiveKit webhook events
- Enrich voice join/leave logs with username, remote addr, quality,
  channel occupancy
- Add channel_id to voice control debug logs

Phase 1 — Client log persistence:
- New logPersistence.ts: rotating JSONL files in appLogDir with
  5-day retention, 2s debounced flush, append mode
- Wire into app startup with flush on beforeunload
- Scope all new FS capabilities to $APPLOG/**

Phase 1 — Rust proxy logging:
- Replace eprintln! with structured log crate (info/warn/error/debug)
  in livekit_proxy.rs and ws_proxy.rs
- Add env_logger with try_init for safe initialization
- Log TLS handshakes, TOFU checks, connection lifecycle, byte counts

Phase 1 — Cache management UI:
- Add Clear Image Cache, Clear Log Files, and Clear All Cache & Restart
  buttons to Settings > Advanced with confirmation dialog

Phase 2 — LiveKit ICE and lifecycle logging:
- Log ICE candidate types (host/srflx/relay) and selected candidate pair
  on every voice connect and auto-reconnect
- Add room lifecycle event handlers: Reconnecting, Reconnected,
  SignalReconnecting, MediaDevicesError, ConnectionQualityChanged
- Expose ICE connection state in getSessionDebugInfo()

Phase 2 — WebSocket reconnection logging:
- Structured reconnection logs with host, attempt, lastSeq
- Log reconnect success with attempt count
- Detailed connection state transitions (open/close with context)

Phase 2 — Server diagnostics endpoint:
- GET /api/v1/diagnostics/connectivity (auth required)
- Returns server info, LiveKit health/URL/node_ip, client remote_addr,
  and private network detection
This commit is contained in:
jevb
2026-03-28 20:42:37 +01:00
parent a1f105c595
commit f30d267fda
20 changed files with 900 additions and 29 deletions
+89
View File
@@ -0,0 +1,89 @@
package api
import (
"net/http"
"runtime"
"time"
"github.com/owncord/server/config"
"github.com/owncord/server/ws"
)
// diagnosticsResponse is returned by GET /api/v1/diagnostics/connectivity.
type diagnosticsResponse struct {
Server serverDiag `json:"server"`
Voice voiceDiag `json:"voice"`
Client clientDiag `json:"client"`
}
type serverDiag struct {
Version string `json:"version"`
Uptime int64 `json:"uptime_s"`
GoVersion string `json:"go_version"`
OnlineUsers int `json:"online_users"`
}
type voiceDiag struct {
Enabled bool `json:"enabled"`
LiveKitURL string `json:"livekit_url,omitempty"`
LiveKitHealth bool `json:"livekit_health"`
NodeIP string `json:"node_ip,omitempty"`
ProxyPath string `json:"proxy_path"`
}
type clientDiag struct {
RemoteAddr string `json:"remote_addr"`
IsPrivateNet bool `json:"is_private_network"`
}
func handleDiagnosticsConnectivity(
cfg *config.Config,
ver string,
hub *ws.Hub,
) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
clientAddr := clientIP(r)
lkHealthy := false
if ok, _ := hub.LiveKitHealthCheck(); ok {
lkHealthy = true
}
resp := diagnosticsResponse{
Server: serverDiag{
Version: ver,
Uptime: int64(time.Since(serverStartTime).Seconds()),
GoVersion: runtime.Version(),
OnlineUsers: hub.ClientCount(),
},
Voice: voiceDiag{
Enabled: cfg.Voice.LiveKitURL != "",
LiveKitURL: cfg.Voice.LiveKitURL,
LiveKitHealth: lkHealthy,
NodeIP: cfg.Voice.NodeIP,
ProxyPath: "/livekit",
},
Client: clientDiag{
RemoteAddr: clientAddr,
IsPrivateNet: isPrivateIP(clientAddr),
},
}
writeJSON(w, http.StatusOK, resp)
}
}
// isPrivateIP checks if an IP string is in a private/reserved range.
func isPrivateIP(ip string) bool {
for _, prefix := range []string{
"10.", "172.16.", "172.17.", "172.18.", "172.19.",
"172.20.", "172.21.", "172.22.", "172.23.", "172.24.",
"172.25.", "172.26.", "172.27.", "172.28.", "172.29.",
"172.30.", "172.31.", "192.168.", "127.", "::1", "fc", "fd",
} {
if len(ip) >= len(prefix) && ip[:len(prefix)] == prefix {
return true
}
}
return false
}