Building a Custom Search Algorithm with Elasticsearch in Shopware 6.7

Introduction#

Shopware 6.7 represents a significant leap forward in e-commerce platform capabilities, particularly in the realm of search functionality. With the integration of Elasticsearch as the primary search engine, developers now have unprecedented opportunities to create sophisticated custom search algorithms that can dramatically improve product discoverability and user experience.

In this technical deep dive, we'll explore how to build a custom search algorithm leveraging Elasticsearch within Shopware 6.7's robust architecture. We'll cover the underlying mechanisms, implementation strategies, and performance optimization techniques that will help you create search experiences that go beyond traditional keyword matching.

Understanding Shopware 6.7 Search Architecture#

Before diving into custom implementations, it's crucial to understand how Shopware 6.7 handles search internally. The platform utilizes Elasticsearch as its primary search engine, which provides powerful full-text search capabilities, faceted filtering, and real-time indexing.

The search pipeline in Shopware 6.7 consists of several key components:

1. Elasticsearch Indexing: Product data is indexed into Elasticsearch documents
2. Search Query Processing: Custom query builders translate user input into Elasticsearch queries
3. Result Scoring: Elasticsearch's relevance scoring algorithm determines result order
4. Faceted Filtering: Advanced filtering capabilities based on product attributes

Setting Up Custom Search Components#

To build a custom search algorithm, we need to extend Shopware 6.7's existing search infrastructure. The first step involves creating a custom search service that can intercept and modify the standard search behavior.

<?php
// src/Core/Content/Product/Search/ProductSearchService.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product\Search;

use Shopware\Core\Content\Product\ProductCollection;
use Shopware\Core\Content\Product\ProductDefinition;
use Shopware\Core\Framework\DataAbstractionLayer\Search\Criteria;
use Shopware\Core\Framework\DataAbstractionLayer\Search\Filter\EqualsFilter;
use Elasticsearch\ClientBuilder;
use Elasticsearch\Client;

class ProductSearchService
{
    private Client $client;
    
    public function __construct(Client $client)
    {
        $this->client = $client;
    }
    
    public function customSearch(array $searchTerms, array $filters = []): ProductCollection
    {
        // Build custom Elasticsearch query
        $query = $this->buildCustomQuery($searchTerms, $filters);
        
        $params = [
            'index' => 'product',
            'body' => $query
        ];
        
        $response = $this->client->search($params);
        
        return $this->mapResultsToProducts($response);
    }
    
    private function buildCustomQuery(array $terms, array $filters): array
    {
        $boolQuery = [
            'must' => [],
            'should' => [],
            'filter' => []
        ];
        
        // Process search terms with custom logic
        foreach ($terms as $term) {
            $boolQuery['must'][] = [
                'multi_match' => [
                    'query' => $term,
                    'fields' => ['name^2', 'description', 'keywords'],
                    'type' => 'bool_prefix',
                    'boost' => 1.0
                ]
            ];
        }
        
        // Add custom boosting logic
        if (!empty($filters['category'])) {
            $boolQuery['filter'][] = [
                'terms' => [
                    'categories.id' => $filters['category']
                ]
            ];
        }
        
        return [
            'query' => [
                'bool' => $boolQuery
            ],
            'size' => 100,
            'sort' => [
                '_score',
                ['createdAt' => ['order' => 'desc']]
            ]
        ];
    }
}

Advanced Search Algorithm Implementation#

The heart of our custom search algorithm lies in the sophisticated query construction and result scoring mechanisms. Let's examine how we can implement advanced ranking strategies that go beyond Elasticsearch's default scoring.

Semantic Search Integration#

One powerful enhancement involves integrating semantic search capabilities into traditional keyword matching:

<?php
// src/Core/Content/Product/Search/SemanticSearchService.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product\Search;

class SemanticSearchService
{
    private array $semanticMappings = [];
    
    public function __construct()
    {
        // Load semantic mappings from configuration or external service
        $this->loadSemanticMappings();
    }
    
    public function enhanceQueryWithSemantics(array $originalQuery, string $searchTerm): array
    {
        $enhancedQuery = $originalQuery;
        
        // Find semantically related terms
        $semanticTerms = $this->findSemanticEquivalents($searchTerm);
        
        if (!empty($semanticTerms)) {
            $enhancedQuery['query']['bool']['should'][] = [
                'terms' => [
                    'keywords' => $semanticTerms,
                    'boost' => 2.0
                ]
            ];
            
            // Add semantic synonyms to multi-match query
            foreach ($enhancedQuery['query']['bool']['must'] as &$mustClause) {
                if (isset($mustClause['multi_match'])) {
                    $mustClause['multi_match']['fields'][] = 'keywords^1.5';
                }
            }
        }
        
        return $enhancedQuery;
    }
    
    private function findSemanticEquivalents(string $term): array
    {
        // This would typically call an external NLP service or use a pre-built mapping
        $mappings = [
            'sneakers' => ['running shoes', 'athletic footwear', 'trainer'],
            'phone' => ['mobile', 'smartphone', 'cell phone'],
            'laptop' => ['notebook', 'computer', 'laptop computer']
        ];
        
        return $mappings[$term] ?? [];
    }
}

Dynamic Relevance Scoring#

Custom relevance scoring can dramatically improve search quality by considering business-specific factors:

<?php
// src/Core/Content/Product/Search/RelevanceScorer.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product\Search;

class RelevanceScorer
{
    public function calculateProductScore(array $product, array $searchContext): float
    {
        $score = 0.0;
        
        // Base score from Elasticsearch relevance
        $elasticScore = $product['_score'] ?? 1.0;
        $score += $elasticScore * 0.3;
        
        // Category priority boost
        $categoryBoost = $this->getCategoryBoost($product['categories'] ?? [], $searchContext['category']);
        $score += $categoryBoost * 0.2;
        
        // Price competitiveness factor
        $priceFactor = $this->calculatePriceFactor($product['price'] ?? 0, $searchContext['budget']);
        $score += $priceFactor * 0.2;
        
        // Product popularity (sales count, reviews)
        $popularityScore = $this->calculatePopularityScore($product);
        $score += $popularityScore * 0.3;
        
        return $score;
    }
    
    private function getCategoryBoost(array $productCategories, ?string $searchCategory): float
    {
        if (!$searchCategory || empty($productCategories)) {
            return 1.0;
        }
        
        foreach ($productCategories as $category) {
            if ($category['id'] === $searchCategory) {
                return 2.0; // Direct category match
            }
            
            if (isset($category['parentId']) && $category['parentId'] === $searchCategory) {
                return 1.5; // Parent category match
            }
        }
        
        return 1.0;
    }
    
    private function calculatePriceFactor(float $productPrice, ?float $userBudget): float
    {
        if (!$userBudget) {
            return 1.0;
        }
        
        $priceRatio = $productPrice / $userBudget;
        
        if ($priceRatio <= 0.5) {
            return 1.8; // Under budget - highly relevant
        } elseif ($priceRatio <= 1.0) {
            return 1.5; // Within budget - relevant
        } elseif ($priceRatio <= 2.0) {
            return 1.2; // Slightly over budget - somewhat relevant
        }
        
        return 0.8; // Over budget - less relevant
    }
    
    private function calculatePopularityScore(array $product): float
    {
        $salesCount = $product['sales'] ?? 0;
        $reviewCount = $product['reviews'] ?? 0;
        $rating = $product['avgRating'] ?? 0.0;
        
        // Normalize scores (assuming max values)
        $normalizedSales = min($salesCount / 1000, 1.0);
        $normalizedReviews = min($reviewCount / 50, 1.0);
        $normalizedRating = min($rating / 5.0, 1.0);
        
        return ($normalizedSales * 0.3 + $normalizedReviews * 0.4 + $normalizedRating * 0.3) * 10;
    }
}

Performance Optimization Strategies#

Building custom search algorithms in Shopware 6.7 requires careful attention to performance considerations. Here are key optimization techniques:

Query Caching and Memoization#

<?php
// src/Core/Content/Product/Search/CachedSearchService.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product\Search;

class CachedSearchService extends ProductSearchService
{
    private array $cache = [];
    private int $cacheTtl = 300; // 5 minutes
    
    public function customSearch(array $searchTerms, array $filters = []): ProductCollection
    {
        $cacheKey = md5(serialize(['terms' => $searchTerms, 'filters' => $filters]));
        
        if (isset($this->cache[$cacheKey]) && 
            ($this->cache[$cacheKey]['timestamp'] + $this->cacheTtl) > time()) {
            return $this->cache[$cacheKey]['result'];
        }
        
        $result = parent::customSearch($searchTerms, $filters);
        
        $this->cache[$cacheKey] = [
            'result' => $result,
            'timestamp' => time()
        ];
        
        return $result;
    }
}

Index Optimization#

Proper index configuration is crucial for search performance. Custom analyzers and mappings can significantly impact query speed:

# config/elasticsearch/product_mapping.yml

product:
  properties:
    name:
      type: text
      analyzer: custom_search_analyzer
      fields:
        keyword:
          type: keyword
    description:
      type: text
      analyzer: custom_search_analyzer
    keywords:
      type: text
      analyzer: custom_search_analyzer
    categories:
      type: nested
      properties:
        id:
          type: keyword
        name:
          type: text
          analyzer: custom_search_analyzer

settings:
  analysis:
    analyzer:
      custom_search_analyzer:
        type: custom
        tokenizer: standard
        filter:
          - lowercase
          - stop
          - ngram_filter
          - edge_ngram_filter

Integration with Shopware 6.7 Core#

To seamlessly integrate our custom search algorithm into Shopware 6.7, we need to override the existing search service using dependency injection:

<?php
// src/Core/Content/Product/ProductSearchService.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product;

use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;

class ProductSearchService extends \Shopware\Core\Content\Product\ProductSearchService
{
    private CustomSearchService $customSearchService;
    
    public function __construct(
        CustomSearchService $customSearchService,
        // ... other dependencies
    ) {
        $this->customSearchService = $customSearchService;
        parent::__construct(/* ... */);
    }
    
    public function search(Criteria $criteria): ProductCollection
    {
        // Check if custom search should be applied
        if ($this->shouldUseCustomSearch($criteria)) {
            return $this->customSearchService->customSearch(
                $this->extractSearchTerms($criteria),
                $this->extractFilters($criteria)
            );
        }
        
        return parent::search($criteria);
    }
    
    private function shouldUseCustomSearch(Criteria $criteria): bool
    {
        // Custom logic to determine when to use custom search
        return !empty($criteria->getTerm());
    }
}

Monitoring and Analytics#

Implementing comprehensive monitoring is essential for maintaining optimal search performance. Our custom algorithm should include built-in analytics:

<?php
// src/Core/Content/Product/Search/SearchAnalytics.php

declare(strict_types=1);

namespace MyPlugin\Core\Content\Product\Search;

class SearchAnalytics
{
    public function logSearchQuery(string $query, array $results, array $context): void
    {
        // Log search queries for analytics and improvement
        $logData = [
            'timestamp' => time(),
            'query' => $query,
            'results_count' => count($results),
            'execution_time' => $this->calculateExecutionTime(),
            'user_context' => $context,
            'search_type' => 'custom'
        ];
        
        // Store in database or external analytics service
        $this->storeLog($logData);
    }
    
    public function getSearchPerformanceMetrics(): array
    {
        // Return performance metrics for monitoring
        return [
            'avg_response_time' => $this->calculateAverageResponseTime(),
            'search_accuracy' => $this->calculateSearchAccuracy(),
            'user_engagement' => $this->calculateUserEngagement()
        ];
    }
}

Conclusion#

Building custom search algorithms with Elasticsearch in Shopware 6.7 opens up tremendous possibilities for creating highly personalized and effective product discovery experiences. By understanding the underlying architecture, implementing sophisticated ranking strategies, and optimizing performance through caching and indexing, developers can create search experiences that significantly outperform traditional approaches.

The key to success lies in balancing algorithmic sophistication with performance considerations, ensuring that custom search enhancements provide tangible business value without compromising user experience. As e-commerce continues to evolve, these advanced search capabilities will become increasingly critical for maintaining competitive advantage in product discovery and customer engagement.

Remember to thoroughly test your custom algorithms against real-world data, monitor performance metrics, and continuously iterate based on user behavior analytics to ensure optimal results. The investment in custom search implementation pays dividends through improved conversion rates, enhanced user satisfaction, and more effective product recommendations.