Add RSS sync for Lidarr

Dockerfile

@@ -1,13 +1,10 @@
-# Use the official Microsoft .NET Core SDK image as the build environment
 FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
 WORKDIR /app
 
-# Copy everything and build the project
 COPY . ./
 RUN dotnet restore
 RUN dotnet publish -c Release -o out
 
-# Generate the runtime image
 FROM mcr.microsoft.com/dotnet/aspnet:8.0
 WORKDIR /app
 COPY --from=build-env /app/out .

README.md

@@ -1,25 +1,92 @@
 # UmlautAdaptarr
 
-## Testversion kommt in wenigen Tagen
+## English description coming soon
 
-### Kleine Preview
+## 12.02.2024: Erste Testversion
+Wer möchte kann den UmlautAdaptarr jetzt gerne testen! Über Feedback würde ich mich sehr freuen!
+
+Es sollte mit allen *arrs funktionieren, hat aber nur bei Sonarr schon Auswirkungen (abgesehen vom Caching).
+
+Momentan ist docker dafür nötig, wer kein Docker nutzt muss sich noch etwas gedulden. 
+
+[Link zum Docker Image](https://hub.docker.com/r/pcjones/umlautadaptarr)
+
+Zusätzlich müsst ihr in Sonarr oder Prowlarr einen neuen Indexer hinzufügen (für jeden Indexer, bei dem UmlautAdapdarr greifen soll).
+
+Am Beispiel von sceneNZBs:
+
+![grafik](https://github.com/PCJones/UmlautAdaptarr/assets/377223/07c7ca45-e0e5-4a82-af63-365bb23c55c9)
+
+Also alles wie immer, nur dass ihr als API-URL nicht direkt z.B. `https://scenenzbs.com` eingebt, sondern 
+`http://localhost:5005/_/scenenzbs.com`
+
+Den API-Key müsst ihr natürlich auch ganz normal setzen.
+
+## Was macht UmlautAdaptarr überhaupt?
+UmlautAdaptarr löst mehrere Probleme:
+- Releases mit Umlauten werden grundsätzlich nicht korrekt von den *Arrs importiert
+- Releases mit Umlauten werden oft nicht korrekt gefunden (*Arrs suchen nach "o" statt "ö" & es fehlt häufig die korrekte Zuordnung zur Serie/zum Film beim Indexer)
+- Sonarr & Radarr erwarten immer den englischen Titel von https://thetvdb.com/ bzw. https://www.themoviedb.org/. Das führt bei deutschen Produktionen oder deutschen Übersetzungen oft zu Problemen - falls die *arrs schon mal etwas mit der Meldung `Found matching series/movie via grab history, but release was matched to series by ID. Automatic import is not possible/` nicht importiert haben, dann war das der Grund.
+
+# Wie macht UmlautAdaptarr das?
+UmlautAdaptarr tut so, als wäre es ein Indexer. In Wahrheit schaltet sich UmlautAdaptarr aber nur zwischen die *arrs und den echten Indexer und kann somit die Suchen sowie die Ergebnisse abfangen und bearbeiten.
+Am Ende werden die gefundenen Releases immer so umbenannt, das die Arrs sie einwandfrei erkennen.
+Einige Beispiele findet ihr unter Features.
+
+
+## Features
+
+| Feature                                                           | Status        |
+|-------------------------------------------------------------------|---------------|
+| Sonarr & Prowlarr Support                                         | ✓             |
+| Lidarr Support                                                    | (✓) kein RSS Sync|
+| Releases mit deutschem Titel werden erkannt   | ✓             |
+| Releases mit TVDB-Alias Titel werden erkannt  | ✓             |
+| Korrekte Suche und Erkennung von Titel mit Umlauten                            | ✓             |
+| Anfragen-Caching für 5 Minuten zur Reduzierung der API-Zugriffe   | ✓             |
+| Radarr Support                                                    | Geplant       |
+| Readarr Support                                                   | Geplant       |
+| Prowlarr Unterstützung für "DE" SceneNZBs Kategorien              | Geplant       |
+| Unterstützung weiterer Sprachen neben Deutsch                     | Geplant       |
+| Wünsche?                                                          | Vorschläge?   |
+
+## Beispiel-Funktionalität
 In den Klammern am Ende des Releasenamens (Bild 2 & 4) steht zu Anschauungszwecken der deutsche Titel der vorher nicht gefunden bzw. akzeptiert wurde. Das bleibt natürlich nicht so ;)
 
-**Vorher:**  
+**Vorher:** Release wird zwar gefunden, kann aber kann nicht zu geordnet werden.
 ![Vorherige Suche ohne deutsche Titel](https://i.imgur.com/7pfRzgH.png)  
-Release wird zwar gefunden, kann aber kann nicht zu geordnet werden.
 
-**Jetzt:**  
+**Jetzt:** 2-3 weitere Releases werden gefunden, außerdem meckert Sonarr nicht mehr über den Namen und würde es bei einer automatischen Suche ohne Probleme importieren.
 ![Jetzige Suche mit deutschen Titeln](https://i.imgur.com/k55YIN9.png)  
-2-3 weitere Releases werden gefunden, außerdem meckert Sonarr nicht mehr über den Namen und würde es bei einer automatischen Suche ohne Probleme importieren.
 
-**Vorher:**  
+**Vorher:** Es werden nur Releases mit dem englischen Titel der Serie gefunden
 ![Vorherige Suche, englische Titel](https://i.imgur.com/pbRlOeX.png)  
-Es werden nur Releases mit dem englischen Titel der Serie gefunden
 
-**Jetzt:**  
+**Jetzt:**  Es werden auch Titel mit dem deutschen Namen gefunden :D (haben nicht alle Suchergebnisse auf den Screenshot gepasst)
 ![Jetzige Suche, deutsche und englische Titel](https://i.imgur.com/eeq0Voj.png)  
-Es werden auch Titel mit dem deutschen Namen gefunden :D (haben nicht alle Suchergebnisse auf den Screenshot gepasst)
 
-PS:
+**Vorher:** Die deutsche Produktion `Alone - Überlebe die Wildnis` hat auf [TheTVDB](https://thetvdb.com/series/alone-uberlebe-die-wildnis) den Englischen Namen `Alone Germany`.
-Das Problem, dass Prowlarr mit SceneNZBs nicht richtig funktioniert ist damit auch behoben :D
+
+Sonarr erwartet immer den Englischen Namen, der hier natürlich nicht gegeben ist.
+![grafik](https://github.com/PCJones/UmlautAdaptarr/assets/377223/62158f77-ecc2-4747-af85-4b8f94f51ab4)
+
+**Jetzt:** UmlautAdaptarr hat die Releases in `Alone Germany` umbenannt und Sonarr hat keine Probleme mehr
+![grafik](https://github.com/PCJones/UmlautAdaptarr/assets/377223/57539ffc-b8a6-4255-a7f8-03079c10b1e8)
+
+**Vorher:** Hier wird der komplette deutsche Titel im Release angegeben (also mit `- Das Lied von Eis und Feuer`) - glücklicherweise stellt uns [TheTVDB](https://thetvdb.com/series/game-of-thrones) aber diesen längeren Titel als Alias zur Verfügung - nur nutzt Sonarr diese Informationen (bisher) einfach nicht.
+![grafik](https://github.com/PCJones/UmlautAdaptarr/assets/377223/8f3297bd-ebe4-42de-b4e6-952882c8b902)
+
+**Jetzt:** UmlautAdapatarr erkennt alle auf TheTVDB angegebenen Aliase und benennt das Release in den Englischen Titel um
+![grafik](https://github.com/PCJones/UmlautAdaptarr/assets/377223/52f0caf5-6e9d-442e-9018-ba29f954a890)
+
+## Kontakt & Support
+- Öffne gerne ein Issue auf GitHub falls du Unterstützung benötigst.
+- [Telegram](https://t.me/pc_jones)
+- Discord: pcjones1
+- Reddit: /u/IreliaIsLife
+
+
+### Licenses & Metadata source
+- TV Metadata source: https://thetvdb.com
+- Movie Metadata source: https://themoviedb.org
+- Licenses: TODO

UmlautAdaptarr/Controllers/SearchController.cs

@@ -1,7 +1,5 @@
 using Microsoft.AspNetCore.Mvc;
-using Newtonsoft.Json.Linq;
 using System.Text;
-using System.Xml.Linq;
 using UmlautAdaptarr.Models;
 using UmlautAdaptarr.Services;
 using UmlautAdaptarr.Utilities;
@@ -34,7 +32,7 @@ namespace UmlautAdaptarr.Controllers
                 // Rename titles in the single search content
                 if (!string.IsNullOrEmpty(initialSearchResult?.Content))
                 {
-                    inititalProcessedContent = ProcessContent(initialSearchResult.Content, searchItem?.TitleMatchVariations, searchItem?.ExpectedTitle);
+                    inititalProcessedContent = ProcessContent(initialSearchResult.Content, searchItem);
                 }
 
                 var additionalTextSearch = searchItem != null
@@ -76,7 +74,7 @@ namespace UmlautAdaptarr.Controllers
                     }
 
                     // Handle multiple search requests based on German title variations
-                    var aggregatedResult = await AggregateSearchResults(domain, queryParameters, titleSearchVariations, searchItem.TitleMatchVariations, expectedTitle);
+                    var aggregatedResult = await AggregateSearchResults(domain, queryParameters, titleSearchVariations, searchItem);
                     aggregatedResult.AggregateItems(inititalProcessedContent);
 
                     return Content(aggregatedResult.Content, aggregatedResult.ContentType, aggregatedResult.ContentEncoding);
@@ -109,17 +107,17 @@ namespace UmlautAdaptarr.Controllers
         }
 
 
-        private string ProcessContent(string content, string[]? titleMatchVariations = null, string? expectedTitle = null)
+        private string ProcessContent(string content, SearchItem? searchItem)
         {
-            return titleMatchingService.RenameTitlesInContent(content, titleMatchVariations, expectedTitle);
+            return titleMatchingService.RenameTitlesInContent(content, searchItem);
         }
 
         public async Task<AggregatedSearchResult> AggregateSearchResults(
             string domain,
             IDictionary<string, string> queryParameters,
             IEnumerable<string> titleSearchVariations,
-            string[] titleMatchVariations,
+            SearchItem? searchItem
-            string expectedTitle)
+            )
         {
             string defaultContentType = "application/xml";
             Encoding defaultEncoding = Encoding.UTF8;
@@ -143,7 +141,7 @@ namespace UmlautAdaptarr.Controllers
                 }
 
                 // Process and rename titles in the content
-                content = ProcessContent(content, titleMatchVariations, expectedTitle);
+                content = ProcessContent(content, searchItem);
 
                 // Aggregate the items into a single document
                 aggregatedResult.AggregateItems(content);
@@ -157,6 +155,8 @@ namespace UmlautAdaptarr.Controllers
                                   TitleMatchingService titleMatchingService,
                                   SearchItemLookupService searchItemLookupService) : SearchControllerBase(proxyService, titleMatchingService)
     {
+        public readonly string[] AUDIO_CATEGORY_IDS = ["3000", "3010", "3020", "3040", "3050"];
+
         [HttpGet]
         public async Task<IActionResult> MovieSearch([FromRoute] string options, [FromRoute] string domain)
         {
@@ -169,10 +169,27 @@ namespace UmlautAdaptarr.Controllers
         [HttpGet]
         public async Task<IActionResult> GenericSearch([FromRoute] string options, [FromRoute] string domain)
         {
+
         var queryParameters = HttpContext.Request.Query.ToDictionary(
                  q => q.Key,
                  q => string.Join(",", q.Value));
-            return await BaseSearch(options, domain, queryParameters);
+
+            SearchItem? searchItem = null;
+
+            if (queryParameters.TryGetValue("q", out string? title) && !string.IsNullOrEmpty(title))
+            {
+                if (queryParameters.TryGetValue("cat", out string? categories) && !string.IsNullOrEmpty(categories))
+                {
+                    // Search for audio
+                    if (categories.Split(',').Any(category => AUDIO_CATEGORY_IDS.Contains(category)))
+                    {
+                        var mediaType = "audio";
+                        searchItem = await searchItemLookupService.GetOrFetchSearchItemByExternalId(mediaType, title.ToLower());
+                    }
+                }
+            }
+
+            return await BaseSearch(options, domain, queryParameters, searchItem);
         }
 
         [HttpGet]
@@ -192,7 +209,7 @@ namespace UmlautAdaptarr.Controllers
                  q => string.Join(",", q.Value));
 
             SearchItem? searchItem = null;
-            string mediaType = "tv";
+            var mediaType = "tv";
 
             if (queryParameters.TryGetValue("tvdbid", out string? tvdbId) && !string.IsNullOrEmpty(tvdbId))
             {

UmlautAdaptarr/Models/SearchItem.cs

@@ -1,4 +1,5 @@
-using System.Text.RegularExpressions;
+using Microsoft.Extensions.Logging.Abstractions;
+using System.Text.RegularExpressions;
 using UmlautAdaptarr.Utilities;
 
 namespace UmlautAdaptarr.Models
@@ -8,24 +9,51 @@ namespace UmlautAdaptarr.Models
         public int ArrId { get; set; }
         public string ExternalId { get; set; }
         public string Title { get; set; }
-        public bool HasGermanUmlaut => Title?.HasGermanUmlauts() ?? false;
+        public bool HasUmlaut => Title?.HasUmlauts() ?? false;
         public string ExpectedTitle { get; set; }
+        public string? ExpectedAuthor { get; set; }
         public string? GermanTitle { get; set; }
         public string[] TitleSearchVariations { get; set; }
         public string[] TitleMatchVariations { get; set; }
+        public string[] AuthorMatchVariations { get; set; }
         public string MediaType { get; set; }
         // TODO public MediaType instead of string
 
-        public SearchItem(int arrId, string externalId, string title, string expectedTitle, string? germanTitle, string mediaType, string[]? aliases)
+        public SearchItem(
+            int arrId,
+            string externalId,
+            string title,
+            string expectedTitle,
+            string? germanTitle,
+            string mediaType,
+            string[]? aliases,
+            string? expectedAuthor = null)
         {
             ArrId = arrId;
             ExternalId = externalId;
             Title = title;
             ExpectedTitle = expectedTitle;
+            ExpectedAuthor = expectedAuthor;
             GermanTitle = germanTitle;
-            TitleSearchVariations = GenerateTitleVariations(germanTitle).ToArray();
             MediaType = mediaType;
-
+            if (mediaType == "audio" && expectedAuthor != null)
+            {
+                // e.g. Die Ärzte - best of die Ärzte
+                if (expectedTitle.Contains(expectedAuthor))
+                {
+                    var titleWithoutAuthorName = expectedTitle.Replace(expectedAuthor, string.Empty).RemoveExtraWhitespaces().Trim();
+                    TitleMatchVariations = GenerateVariations(titleWithoutAuthorName, mediaType).ToArray();
+                }
+                else
+                {
+                    TitleMatchVariations = GenerateVariations(expectedTitle, mediaType).ToArray();
+                }
+                TitleSearchVariations = GenerateVariations($"{expectedAuthor} {expectedTitle}", mediaType).ToArray();
+                AuthorMatchVariations = GenerateVariations(expectedAuthor, mediaType).ToArray();
+            }
+            else
+            {
+                TitleSearchVariations = GenerateVariations(germanTitle, mediaType).ToArray();
                 var allTitleVariations = new List<string>(TitleSearchVariations);
 
                 // If aliases are not null, generate variations for each and add them to the list
@@ -34,20 +62,22 @@ namespace UmlautAdaptarr.Models
                 {
                     foreach (var alias in aliases)
                     {
-                    allTitleVariations.AddRange(GenerateTitleVariations(alias));
+                        allTitleVariations.AddRange(GenerateVariations(alias, mediaType));
                     }
                 }
 
                 TitleMatchVariations = allTitleVariations.Distinct().ToArray();
+                AuthorMatchVariations = [];
+            }
         }
 
-        private IEnumerable<string> GenerateTitleVariations(string? germanTitle)
+        private IEnumerable<string> GenerateVariations(string? germanTitle, string mediaType)
         {
             if (germanTitle == null)
             {
                 return [];
             }
-            var cleanTitle = germanTitle.RemoveAccentButKeepGermanUmlauts();
+            var cleanTitle = germanTitle.RemoveAccentButKeepGermanUmlauts().GetCleanTitle();
 
             // Start with base variations including handling umlauts
             var baseVariations = new List<string>
@@ -76,13 +106,17 @@ namespace UmlautAdaptarr.Models
                 });
             }
 
+            // If a german title starts with der/die/das also accept variations without it
+            if (mediaType != "audio" && cleanTitle.StartsWith("Der") || cleanTitle.StartsWith("Die") || cleanTitle.StartsWith("Das"))
+            {
+                var cleanTitleWithoutArticle = germanTitle[3..].Trim();
+                baseVariations.AddRange(GenerateVariations(cleanTitleWithoutArticle, mediaType));
+            }
+
             // Remove multiple spaces
-            var cleanedVariations = baseVariations.Select(variation => MultipleWhitespaceRegex().Replace(variation, " "));
+            var cleanedVariations = baseVariations.Select(variation => variation.RemoveExtraWhitespaces());
 
             return cleanedVariations.Distinct();
         }
-
-        [GeneratedRegex(@"\s+")]
-        private static partial Regex MultipleWhitespaceRegex();
     }
 }

UmlautAdaptarr/Program.cs

@@ -1,3 +1,4 @@
+using Microsoft.Extensions.Configuration;
 using System.Net;
 using UmlautAdaptarr.Providers;
 using UmlautAdaptarr.Routing;
@@ -10,8 +11,6 @@ internal class Program
         // TODO:
         // add option to sort by nzb age
 
-        // TODO
-        // add delay between requests
     
        var builder = WebApplication.CreateBuilder(args);
 
@@ -33,12 +32,25 @@ internal class Program
             //options.SizeLimit = 20000;
         });
 
+
+        // TODO workaround to not log api keys
+        builder.Logging.AddFilter((category, level) =>
+        {
+            // Prevent logging of HTTP request and response if the category is HttpClient
+            if (category.Contains("System.Net.Http.HttpClient") || category.Contains("Microsoft.Extensions.Http.DefaultHttpClientFactory"))
+            {
+                return false;
+            }
+            return true;
+        });
+
         builder.Services.AddControllers();
         builder.Services.AddHostedService<ArrSyncBackgroundService>();
-        builder.Services.AddSingleton<TitleApiService>(); // TODO rename
+        builder.Services.AddSingleton<TitleApiService>();
         builder.Services.AddSingleton<SearchItemLookupService>();
         builder.Services.AddSingleton<TitleMatchingService>();
         builder.Services.AddSingleton<SonarrClient>();
+        builder.Services.AddSingleton<LidarrClient>();
         builder.Services.AddSingleton<CacheService>();
         builder.Services.AddSingleton<ProxyService>();
 

UmlautAdaptarr/Providers/LidarrClient.cs

@@ -0,0 +1,146 @@
+using Microsoft.Extensions.Caching.Memory;
+using Newtonsoft.Json;
+using Newtonsoft.Json.Linq;
+using UmlautAdaptarr.Models;
+using UmlautAdaptarr.Services;
+using UmlautAdaptarr.Utilities;
+
+namespace UmlautAdaptarr.Providers
+{
+    public class LidarrClient(
+        IHttpClientFactory clientFactory,
+        IConfiguration configuration,
+        CacheService cacheService,
+        IMemoryCache cache,
+        ILogger<LidarrClient> logger) : ArrClientBase()
+    {
+        private readonly string _lidarrHost = configuration.GetValue<string>("LIDARR_HOST") ?? throw new ArgumentException("LIDARR_HOST environment variable must be set");
+        private readonly string _lidarrApiKey = configuration.GetValue<string>("LIDARR_API_KEY") ?? throw new ArgumentException("LIDARR_API_KEY environment variable must be set");
+        private readonly string _mediaType = "audio";
+
+        public override async Task<IEnumerable<SearchItem>> FetchAllItemsAsync()
+        {
+            var httpClient = clientFactory.CreateClient();
+            var items = new List<SearchItem>();
+
+            try
+            {
+                var lidarrArtistsUrl = $"{_lidarrHost}/api/v1/artist?apikey={_lidarrApiKey}";
+                logger.LogInformation($"Fetching all artists from Lidarr: {UrlUtilities.RedactApiKey(lidarrArtistsUrl)}");
+                var artistsApiResponse = await httpClient.GetStringAsync(lidarrArtistsUrl);
+                var artists = JsonConvert.DeserializeObject<List<dynamic>>(artistsApiResponse);
+
+                if (artists == null)
+                {
+                    logger.LogError($"Lidarr artists API request resulted in null");
+                    return items;
+                }
+                logger.LogInformation($"Successfully fetched {artists.Count} artists from Lidarr.");
+                foreach (var artist in artists)
+                {
+                    var artistId = (int)artist.id;
+
+                    var lidarrAlbumUrl = $"{_lidarrHost}/api/v1/album?artistId={artistId}&apikey={_lidarrApiKey}";
+
+                    if (cache.TryGetValue(lidarrAlbumUrl, out List<dynamic>? albums))
+                    {
+                        logger.LogInformation($"Using cached albums for {UrlUtilities.RedactApiKey(lidarrAlbumUrl)}");
+                    }
+                    else
+                    {
+                        logger.LogInformation($"Fetching all albums from artistId {artistId} from Lidarr: {UrlUtilities.RedactApiKey(lidarrArtistsUrl)}");
+                        var albumApiResponse = await httpClient.GetStringAsync(lidarrAlbumUrl);
+                        albums = JsonConvert.DeserializeObject<List<dynamic>>(albumApiResponse);
+                    }
+
+                    if (albums == null)
+                    {
+                        logger.LogWarning($"Lidarr album API request for artistId {artistId} resulted in null");
+                        continue;
+                    }
+
+                    logger.LogInformation($"Successfully fetched {albums.Count} albums for artistId {artistId} from Lidarr.");
+
+                    // Cache albums for 3 minutes
+                    cache.Set(lidarrAlbumUrl, albums, TimeSpan.FromMinutes(3));
+
+                    foreach (var album in albums)
+                    {
+                        var artistName = (string)album.artist.artistName;
+                        var albumTitle = (string)album.title;
+
+                        var expectedTitle = $"{artistName} {albumTitle}";
+
+                        string[]? aliases = null;
+
+                        // Abuse externalId to set the search string Lidarr uses
+                        var externalId = expectedTitle.RemoveGermanUmlautDots().RemoveAccent().RemoveSpecialCharacters().RemoveExtraWhitespaces().ToLower();
+
+                        var searchItem = new SearchItem
+                        (
+                            arrId: artistId,
+                            externalId: externalId,
+                            title: albumTitle,
+                            expectedTitle: albumTitle,
+                            germanTitle: null,
+                            aliases: aliases,
+                            mediaType: _mediaType,
+                            expectedAuthor: artistName
+                        );
+
+                        items.Add(searchItem);
+                    }
+                }
+
+                logger.LogInformation($"Finished fetching all items from Lidarr");
+            }
+            catch (Exception ex)
+            {
+                logger.LogError($"Error fetching all artists from Lidarr: {ex.Message}");
+            }
+
+            return items;
+        }
+
+        public override async Task<SearchItem?> FetchItemByExternalIdAsync(string externalId)
+        {
+            try
+            {
+                // For now we have to fetch all items every time
+                var searchItems = await FetchAllItemsAsync();
+                foreach (var searchItem in searchItems ?? [])
+                {
+                    try
+                    {
+                        cacheService.CacheSearchItem(searchItem);
+                    }
+                    catch (Exception ex)
+                    {
+                        logger.LogError(ex, $"An error occurred while caching search item with ID {searchItem.ArrId}.");
+                    }
+                }
+            }
+            catch (Exception ex)
+            {
+                logger.LogError($"Error fetching single artist from Lidarr: {ex.Message}");
+            }
+
+            return null;
+        }
+
+        public override async Task<SearchItem?> FetchItemByTitleAsync(string title)
+        {
+            try
+            {
+               // this should never be called at the moment
+                throw new NotImplementedException();
+            }
+            catch (Exception ex)
+            {
+                logger.LogError($"Error fetching single artist from Lidarr: {ex.Message}");
+            }
+
+            return null;
+        }
+    }
+}

UmlautAdaptarr/Providers/SonarrClient.cs

@@ -1,6 +1,4 @@
-using Microsoft.Extensions.Caching.Memory;
+using Newtonsoft.Json;
-using Newtonsoft.Json;
-using System.Net.Http;
 using UmlautAdaptarr.Models;
 using UmlautAdaptarr.Services;
 using UmlautAdaptarr.Utilities;

UmlautAdaptarr/Services/ArrSyncBackgroundService.cs

@@ -13,9 +13,13 @@ namespace UmlautAdaptarr.Services
 {
     public class ArrSyncBackgroundService(
         SonarrClient sonarrClient,
+        LidarrClient lidarrClient,
         CacheService cacheService,
+        IConfiguration configuration,
         ILogger<ArrSyncBackgroundService> logger) : BackgroundService
     {
+        private readonly bool _sonarrEnabled = configuration.GetValue<bool>("SONARR_ENABLED");
+        private readonly bool _lidarrEnabled = configuration.GetValue<bool>("LIDARR_ENABLED");
         protected override async Task ExecuteAsync(CancellationToken stoppingToken)
         {
             logger.LogInformation("ArrSyncBackgroundService is starting.");
@@ -23,43 +27,78 @@ namespace UmlautAdaptarr.Services
             while (!stoppingToken.IsCancellationRequested)
             {
                 logger.LogInformation("ArrSyncBackgroundService is running.");
-                await FetchAndUpdateDataAsync();
+                var syncSuccess = await FetchAndUpdateDataAsync();
                 logger.LogInformation("ArrSyncBackgroundService has completed an iteration.");
 
+                if (syncSuccess)
+                {
                     await Task.Delay(TimeSpan.FromHours(12), stoppingToken);
                 }
+                else
+                {
+                    logger.LogInformation("ArrSyncBackgroundService is sleeping for one hour only because not all syncs were successful.");
+                    await Task.Delay(TimeSpan.FromHours(1), stoppingToken);
+                }
+            }
 
             logger.LogInformation("ArrSyncBackgroundService is stopping.");
         }
 
-        private async Task FetchAndUpdateDataAsync()
+        private async Task<bool> FetchAndUpdateDataAsync()
         {
             try
             {
-                await FetchItemsFromSonarrAsync();
+                var success = true;
+                if (_sonarrEnabled)
+                {
+                    success = await FetchItemsFromSonarrAsync();
+                }
+                if (_lidarrEnabled)
+                {
+                    success = await FetchItemsFromLidarrAsync();
+                }
+                return success;
             }
             catch (Exception ex)
             {
                 logger.LogError(ex, "An error occurred while fetching items from the Arrs.");
             }
+            return false;
         }
 
-        private async Task FetchItemsFromSonarrAsync()
+        private async Task<bool> FetchItemsFromSonarrAsync()
         {
             try
             {
                 var items = await sonarrClient.FetchAllItemsAsync();
                 UpdateSearchItems(items);
+                return items?.Any()?? false;
             }
             catch (Exception ex)
             {
                 logger.LogError(ex, "An error occurred while updating search item from Sonarr.");
             }
+            return false;
         }
 
-        private void UpdateSearchItems(IEnumerable<SearchItem> searchItems)
+        private async Task<bool> FetchItemsFromLidarrAsync()
         {
-            foreach (var searchItem in searchItems)
+            try
+            {
+                var items = await lidarrClient.FetchAllItemsAsync();
+                UpdateSearchItems(items);
+                return items?.Any() ?? false;
+            }
+            catch (Exception ex)
+            {
+                logger.LogError(ex, "An error occurred while updating search item from Lidarr.");
+            }
+            return false;
+        }
+
+        private void UpdateSearchItems(IEnumerable<SearchItem>? searchItems)
+        {
+            foreach (var searchItem in searchItems ?? [])
             {
                 try
                 {

UmlautAdaptarr/Services/CacheService.cs

@@ -1,43 +1,75 @@
-using Microsoft.Extensions.Caching.Memory;
+using Microsoft.AspNetCore.Mvc.Formatters;
+using Microsoft.Extensions.Caching.Memory;
+using System.Reflection.Metadata.Ecma335;
+using System.Text.RegularExpressions;
 using UmlautAdaptarr.Models;
 using UmlautAdaptarr.Utilities;
 
 namespace UmlautAdaptarr.Services
 {
-    public class CacheService(IMemoryCache cache)
+    public partial class CacheService(IMemoryCache cache)
     {
         private readonly Dictionary<string, HashSet<string>> VariationIndex = [];
+        private readonly Dictionary<string, List<(HashSet<string> TitleVariations, string CacheKey)>> AudioVariationIndex = [];
         private const int VARIATION_LOOKUP_CACHE_LENGTH = 5;
 
         public void CacheSearchItem(SearchItem item)
         {
             var prefix = item.MediaType;
-            var normalizedTitle = item.Title.RemoveAccentButKeepGermanUmlauts().ToLower();
+            var cacheKey = $"{prefix}_extid_{item.ExternalId}";
-            // TODO maybe we need to also add the media type (movie/book/show etc)
+            cache.Set(cacheKey, item);
+            if (item.MediaType == "audio")
+            {
+                CacheAudioSearchItem(item, cacheKey);
+                return;
+            }
+
+            var normalizedTitle = item.Title.RemoveAccentButKeepGermanUmlauts().ToLower();
 
-            cache.Set($"{prefix}_extid_{item.ExternalId}", item);
             cache.Set($"{prefix}_title_{normalizedTitle}", item);
 
-            foreach (var variation in item.TitleSearchVariations)
+            foreach (var variation in item.TitleMatchVariations)
             {
-                var normalizedVariation = item.Title.RemoveAccentButKeepGermanUmlauts().ToLower();
+                var normalizedVariation = variation.RemoveAccentButKeepGermanUmlauts().ToLower();
-                var cacheKey = $"{prefix}_var_{normalizedVariation}";
+                cacheKey = $"{prefix}_var_{normalizedVariation}";
                 cache.Set(cacheKey, item);
 
                 // Indexing by prefix
                 var indexPrefix = normalizedVariation[..Math.Min(VARIATION_LOOKUP_CACHE_LENGTH, variation.Length)].ToLower();
                 if (!VariationIndex.ContainsKey(indexPrefix))
                 {
-                    VariationIndex[indexPrefix] = new HashSet<string>();
+                    VariationIndex[indexPrefix] = [];
                 }
                 VariationIndex[indexPrefix].Add(cacheKey);
             }
         }
 
+        public void CacheAudioSearchItem(SearchItem item, string cacheKey)
+        {
+            // Index author and title variations
+            foreach (var authorVariation in item.AuthorMatchVariations)
+            {
+                var normalizedAuthor = authorVariation.NormalizeForComparison();
+
+                if (!AudioVariationIndex.ContainsKey(normalizedAuthor))
+                {
+                    AudioVariationIndex[normalizedAuthor] = [];
+                }
+
+                var titleVariations = item.TitleMatchVariations.Select(titleMatchVariation => titleMatchVariation.NormalizeForComparison()).ToHashSet();
+                AudioVariationIndex[normalizedAuthor].Add((titleVariations, cacheKey));
+            }
+        }
+
         public SearchItem? SearchItemByTitle(string mediaType, string title)
         {
             var normalizedTitle = title.RemoveAccentButKeepGermanUmlauts().ToLower();
 
+            if (mediaType == "audio")
+            {
+                return FindBestMatchForAudio(normalizedTitle.NormalizeForComparison());
+            }
+
             // Use the first few characters of the normalized title for cache prefix search
             var cacheSearchPrefix = normalizedTitle[..Math.Min(VARIATION_LOOKUP_CACHE_LENGTH, normalizedTitle.Length)];
 
@@ -80,10 +112,12 @@ namespace UmlautAdaptarr.Services
         {
             var normalizedTitle = title.RemoveAccentButKeepGermanUmlauts().ToLower();
 
+
             if (mediaType == "generic")
             {
                 // TODO
             }
+
             cache.TryGetValue($"{mediaType}_var_{normalizedTitle}", out SearchItem? item);
             if (item == null)
             {
@@ -91,5 +125,33 @@ namespace UmlautAdaptarr.Services
             }
             return item;
         }
+
+        private SearchItem? FindBestMatchForAudio(string normalizedOriginalTitle)
+        {
+            foreach (var authorEntry in AudioVariationIndex)
+            {
+                if (normalizedOriginalTitle.Contains(authorEntry.Key))
+                {
+                    var sortedEntries = authorEntry.Value.OrderByDescending(entry => entry.TitleVariations.FirstOrDefault()?.Length).ToList();
+
+                    foreach (var (titleVariations, cacheKey) in sortedEntries)
+                    {
+                        if (titleVariations.Any(normalizedOriginalTitle.Contains))
+                        {
+                            if (cache.TryGetValue(cacheKey, out SearchItem? item))
+                            {
+                                return item;
+                            }
+                        }
+                    }
+                }
+            }
+
+            return null;
+        }
+
+
+        [GeneratedRegex("\\s")]
+        private static partial Regex WhiteSpaceRegex();
     }
 }

UmlautAdaptarr/Services/SearchItemLookupService.cs

@@ -3,8 +3,10 @@ using UmlautAdaptarr.Providers;
 
 namespace UmlautAdaptarr.Services
 {
-    public class SearchItemLookupService(CacheService cacheService, SonarrClient sonarrClient)
+    public class SearchItemLookupService(CacheService cacheService, SonarrClient sonarrClient, LidarrClient lidarrClient, IConfiguration configuration)
     {
+        private readonly bool _sonarrEnabled = configuration.GetValue<bool>("SONARR_ENABLED");
+        private readonly bool _lidarrEnabled = configuration.GetValue<bool>("LIDARR_ENABLED");
         public async Task<SearchItem?> GetOrFetchSearchItemByExternalId(string mediaType, string externalId)
         {
             // Attempt to get the item from the cache first
@@ -19,9 +21,17 @@ namespace UmlautAdaptarr.Services
             switch (mediaType)
             {
                 case "tv":
+                    if (_sonarrEnabled)
+                    {
                         fetchedItem = await sonarrClient.FetchItemByExternalIdAsync(externalId);
+                    }
+                    break;
+                case "audio":
+                    if (_lidarrEnabled)
+                    {
+                        fetchedItem = await lidarrClient.FetchItemByExternalIdAsync(externalId);
+                    }
                     break;
-                    // TODO Add cases for other sources like Radarr, Lidarr, etc.
             }
 
             // If an item is fetched, cache it
@@ -47,7 +57,12 @@ namespace UmlautAdaptarr.Services
             switch (mediaType)
             {
                 case "tv":
+                    if (_sonarrEnabled)
+                    {
                         fetchedItem = await sonarrClient.FetchItemByTitleAsync(title);
+                    }
+                    break;
+                case "audio":
                     break;
                     // TODO add cases for other sources as needed, such as Radarr, Lidarr, etc.
             }
@@ -61,5 +76,4 @@ namespace UmlautAdaptarr.Services
             return fetchedItem;
         }
     }
-
 }

UmlautAdaptarr/Services/TitleMatchingService.cs

@@ -1,17 +1,17 @@
 using System.Text.RegularExpressions;
 using System.Xml.Linq;
+using UmlautAdaptarr.Models;
 using UmlautAdaptarr.Utilities;
 
 namespace UmlautAdaptarr.Services
 {
     public partial class TitleMatchingService(CacheService cacheService, ILogger<TitleMatchingService> logger)
     {
-        public string RenameTitlesInContent(string content, string[]? titleMatchVariations, string? expectedTitle)
+        public string RenameTitlesInContent(string content, SearchItem? searchItem)
         {
             var xDoc = XDocument.Parse(content);
 
-            // If expectedTitle and titleMatchVariations are provided use them, if not use the CacheService to find matches.
+            bool useCacheService = searchItem == null;
-            bool useCacheService = string.IsNullOrEmpty(expectedTitle) || titleMatchVariations?.Length == 0;
 
             foreach (var item in xDoc.Descendants("item"))
             {
@@ -21,33 +21,146 @@ namespace UmlautAdaptarr.Services
                     var originalTitle = titleElement.Value;
                     var normalizedOriginalTitle = NormalizeTitle(originalTitle);
 
-                    if (useCacheService)
-                    {
                     var categoryElement = item.Element("category");
                     var category = categoryElement?.Value;
                     var mediaType = GetMediaTypeFromCategory(category);
+
                     if (mediaType == null)
                     {
                         continue;
                     }
 
-                        // Use CacheService to find a matching SearchItem by title
+                    if (useCacheService)
-                        var searchItem = cacheService.SearchItemByTitle(mediaType, originalTitle);
-                        if (searchItem != null)
                     {
-                            // If a SearchItem is found, use its ExpectedTitle and titleMatchVariations for renaming
+                        // Use CacheService to find a matching SearchItem by title
-                            expectedTitle = searchItem.ExpectedTitle;
+                        searchItem = cacheService.SearchItemByTitle(mediaType, normalizedOriginalTitle);
-                            titleMatchVariations = searchItem.TitleMatchVariations;
                     }
-                        else
+
+                    if (searchItem == null)
                     {
                         // Skip processing this item if no matching SearchItem is found
                         continue;
                     }
+
+                    switch (mediaType)
+                    {
+                        case "tv":
+                            FindAndReplaceForMoviesAndTV(logger, searchItem, titleElement, originalTitle, normalizedOriginalTitle!);
+                            break;
+                        case "movie":
+                            FindAndReplaceForMoviesAndTV(logger, searchItem, titleElement, originalTitle, normalizedOriginalTitle!);
+                            break;
+                        case "audio":
+                            FindAndReplaceForAudio(searchItem, titleElement, originalTitle!);
+                            break;
+                        default:
+                            throw new NotImplementedException();
+                    }
+                }
             }
 
+            return xDoc.ToString();
+        }
+
+        public void FindAndReplaceForAudio(SearchItem searchItem, XElement? titleElement, string originalTitle)
+        {
+            var authorMatch = FindBestMatch(searchItem.AuthorMatchVariations, originalTitle.NormalizeForComparison(), originalTitle);
+            var titleMatch = FindBestMatch(searchItem.TitleMatchVariations, originalTitle.NormalizeForComparison(), originalTitle);
+
+            if (authorMatch.Item1 && titleMatch.Item1)
+            {
+                int matchEndPositionInOriginal = Math.Max(authorMatch.Item3, titleMatch.Item3);
+
+                var test = originalTitle[matchEndPositionInOriginal];
+                // Check and adjust for immediate following delimiter
+                if (matchEndPositionInOriginal < originalTitle.Length && new char[] { ' ', '-', '_', '.' }.Contains(originalTitle[matchEndPositionInOriginal]))
+                {
+                    matchEndPositionInOriginal++; // Skip the delimiter if it's immediately after the match
+                }
+
+                // Ensure we trim any leading delimiters from the suffix
+                string suffix = originalTitle[matchEndPositionInOriginal..].TrimStart([' ', '-', '_', '.']).Trim();
+                suffix = suffix.Replace("-", ".");
+
+                // Concatenate the expected title with the remaining suffix
+                var updatedTitle = $"{searchItem.ExpectedAuthor} - {searchItem.ExpectedTitle}-{suffix}";
+
+                // Update the title element
+                titleElement.Value = updatedTitle;
+                logger.LogInformation($"TitleMatchingService - Title changed: '{originalTitle}' to '{updatedTitle}'");
+            }
+            else
+            {
+                logger.LogInformation("TitleMatchingService - No satisfactory fuzzy match found for both author and title.");
+            }
+        }
+
+
+        private Tuple<bool, int, int> FindBestMatch(string[] variations, string normalizedOriginal, string originalTitle)
+        {
+            bool found = false;
+            int bestStart = int.MaxValue;
+            int bestEndInOriginal = -1;
+
+            foreach (var variation in variations)
+            {
+                var normalizedVariation = variation.NormalizeForComparison();
+                int startNormalized = normalizedOriginal.IndexOf(normalizedVariation);
+
+                if (startNormalized >= 0)
+                {
+                    found = true;
+                    // Map the start position from the normalized string back to the original string
+                    int startOriginal = MapNormalizedIndexToOriginal(normalizedOriginal, originalTitle, startNormalized);
+                    int endOriginal = MapNormalizedIndexToOriginal(normalizedOriginal, originalTitle, startNormalized + normalizedVariation.Length);
+
+                    bestStart = Math.Min(bestStart, startOriginal);
+                    bestEndInOriginal = Math.Max(bestEndInOriginal, endOriginal);
+                }
+            }
+
+            if (!found) return Tuple.Create(false, 0, 0);
+            return Tuple.Create(found, bestStart, bestEndInOriginal);
+        }
+
+        // Maps an index from the normalized string back to a corresponding index in the original string
+        private int MapNormalizedIndexToOriginal(string normalizedOriginal, string originalTitle, int normalizedIndex)
+        {
+            // Count non-special characters up to the given index in the normalized string
+            int nonSpecialCharCount = 0;
+            for (int i = 0; i < normalizedIndex && i < normalizedOriginal.Length; i++)
+            {
+                if (char.IsLetterOrDigit(normalizedOriginal[i]))
+                {
+                    nonSpecialCharCount++;
+                }
+            }
+
+            // Count non-special characters in the original title to find the corresponding index
+            int originalIndex = 0;
+            for (int i = 0; i < originalTitle.Length; i++)
+            {
+                if (char.IsLetterOrDigit(originalTitle[i]))
+                {
+                    if (--nonSpecialCharCount < 0)
+                    {
+                        break;
+                    }
+                }
+                originalIndex = i;
+            }
+
+            return originalIndex;
+        }
+
+        // This method replaces the first variation that starts at the beginning of the release title
+        private static void FindAndReplaceForMoviesAndTV(ILogger<TitleMatchingService> logger, SearchItem searchItem, XElement? titleElement, string originalTitle, string normalizedOriginalTitle)
+        {
+            var titleMatchVariations = searchItem.TitleMatchVariations;
+            var expectedTitle = searchItem.ExpectedTitle;
+            var variationsOrderedByLength = titleMatchVariations!.OrderByDescending(variation => variation.Length);
             // Attempt to find a variation that matches the start of the original title
-                    foreach (var variation in titleMatchVariations!)
+            foreach (var variation in variationsOrderedByLength)
             {
                 // Skip variations that are already the expectedTitle
                 if (variation == expectedTitle)
@@ -56,11 +169,19 @@ namespace UmlautAdaptarr.Services
                 }
 
                 // Variation is already normalized at creation
-                        var pattern = "^" + Regex.Escape(variation).Replace("\\ ", "[._ ]");
+                var variationMatchPattern = "^" + Regex.Escape(variation).Replace("\\ ", "[._ ]");
 
                 // Check if the originalTitle starts with the variation (ignoring case and separators)
-                        if (Regex.IsMatch(normalizedOriginalTitle, pattern, RegexOptions.IgnoreCase))
+                if (Regex.IsMatch(normalizedOriginalTitle, variationMatchPattern, RegexOptions.IgnoreCase))
                 {
+                    // Workaround for the rare case of e.g. "Frieren: Beyond Journey's End" that also has the alias "Frieren"
+                    if (expectedTitle!.StartsWith(variation, StringComparison.OrdinalIgnoreCase))
+                    {
+                        logger.LogWarning($"TitleMatchingService - Didn't rename: '{originalTitle}' because the expected title '{expectedTitle}' starts with the variation '{variation}'");
+                        continue;
+                    }
+                    var originalTitleMatchPattern = "^" + Regex.Escape(variation).Replace("\\ ", "[._ ]");
+
                     // Find the first separator used in the original title for consistent replacement
                     var separator = FindFirstSeparator(originalTitle);
                     // Reconstruct the expected title using the original separator
@@ -96,11 +217,6 @@ namespace UmlautAdaptarr.Services
                 }
             }
         }
-            }
-
-            return xDoc.ToString();
-        }
-
 
         private static string NormalizeTitle(string title)
         {
@@ -117,7 +233,11 @@ namespace UmlautAdaptarr.Services
 
         private static string ReconstructTitleWithSeparator(string title, char separator)
         {
-            // Replace spaces with the original separator found in the title
+            if (separator != ' ')
+            {
+                return title;
+            }
+            
             return title.Replace(' ', separator);
         }
 
@@ -144,6 +264,10 @@ namespace UmlautAdaptarr.Services
             {
                 return "book";
             }
+            else if (category.StartsWith("Audio"))
+            {
+                return "audio";
+            }
 
             return null;
         }
@@ -151,5 +275,6 @@ namespace UmlautAdaptarr.Services
 
         [GeneratedRegex("[._ ]")]
             private static partial Regex WordSeperationCharRegex();
+
     }
 }

UmlautAdaptarr/Services/TitleQueryServiceLegacy.cs

@@ -141,7 +141,7 @@ namespace UmlautAdaptarr.Services
                 germanTitle = titleApiResponseData.germanTitle;
                 hasGermanTitle = true;
 
-                var hasGermanUmlaut = germanTitle?.HasGermanUmlauts() ?? false;
+                var hasGermanUmlaut = germanTitle?.HasUmlauts() ?? false;
 
                 var result = (hasGermanUmlaut, germanTitle, expectedTitle);
                 memoryCache.Set(cacheKey, result, new MemoryCacheEntryOptions

UmlautAdaptarr/Utilities/Extensions.cs

@@ -1,9 +1,10 @@
 using System.Globalization;
 using System.Text;
+using System.Text.RegularExpressions;
 
 namespace UmlautAdaptarr.Utilities
 {
-    public static class Extensions
+    public static partial class Extensions
     {
         public static string GetQuery(this HttpContext context, string key)
         {
@@ -46,6 +47,23 @@ namespace UmlautAdaptarr.Utilities
             return stringBuilder.ToString().Normalize(NormalizationForm.FormC);
         }
 
+        // TODO possibly replace GetCleanTitle with RemoveSpecialCharacters
+        public static string GetCleanTitle(this string text)
+        {
+            return text.Replace("(", "").Replace(")", "").Replace("?","").Replace(":", "").Replace("'", "");
+        }
+
+        public static string NormalizeForComparison(this string text)
+        {
+            return text.RemoveGermanUmlautDots().RemoveAccent().RemoveSpecialCharacters().Replace(" ", "").Trim().ToLower();
+        }
+
+        public static string RemoveSpecialCharacters(this string text)
+        {
+            return SpecialCharactersRegex().Replace(text, "");
+        }
+
+
         public static string ReplaceGermanUmlautsWithLatinEquivalents(this string text)
         {
             return text
@@ -70,11 +88,22 @@ namespace UmlautAdaptarr.Utilities
                 .Replace("ß", "ss");
         }
 
-        public static bool HasGermanUmlauts(this string text)
+        public static string RemoveExtraWhitespaces(this string text)
+        {
+            return MultipleWhitespaceRegex().Replace(text, " ");
+        }
+
+        public static bool HasUmlauts(this string text)
         {
             if (text == null) return false;
             var umlauts = new[] { 'ö', 'ä', 'ü', 'Ä', 'Ü', 'Ö', 'ß' };
             return umlauts.Any(text.Contains);
         }
+
+        [GeneratedRegex("[^a-zA-Z0-9 ]+", RegexOptions.Compiled)]
+        private static partial Regex SpecialCharactersRegex();
+
+        [GeneratedRegex(@"\s+")]
+        private static partial Regex MultipleWhitespaceRegex();
     }
 }

UmlautAdaptarr/libman.json

@@ -0,0 +1,5 @@
+{
+  "version": "1.0",
+  "defaultProvider": "cdnjs",
+  "libraries": []
+}

UmlautAdaptarr/secrets_example.json

@@ -1,4 +1,8 @@
 {
+  "SONARR_ENABLED": false,
   "SONARR_HOST": "http://localhost:8989",
-  "SONARR_API_KEY": ""
+  "SONARR_API_KEY": "",
+  "LIDARR_ENABLED": false,
+  "LIDARR_HOST": "http://localhost:8686",
+  "LIDARR_API_KEY": ""
 }

docker-compose.yml

@@ -1,9 +1,24 @@
 version: '3.8'
 services:
   umlautadaptarr:
-    build: .
+    #uncomment this to get the development branch
+    #build: https://github.com/PCJones/UmlautAdaptarr.git#develop
+    build: https://github.com/PCJones/UmlautAdaptarr.git#master
+    image: umlautadaptarr
+    restart: unless-stopped
     environment:
-      SONARR_HOST: "http://localhost:8989"
+      - TZ=Europe/Berlin
-      SONARR_API_KEY: ""
+      - SONARR_ENABLED=false
+      - SONARR_HOST=http://localhost:8989
+      - SONARR_API_KEY=API_KEY
+      - RADARR_ENABLED=false
+      - RADARR_HOST=http://localhost:7878
+      - RADARR_API_KEY=API_KEY
+      - READARR_ENABLED=false
+      - READARR_HOST=http://localhost:8787
+      - READARR_API_KEY=API_KEY
+      - LIDARR_ENABLED=false
+      - LIDARR_HOST=http://localhost:8686
+      - LIDARR_API_KEY=API_KEY
     ports:
-      - "5005:5005" 
+      - "5005":"5005"