Building Custom Import/Export Profiles in Shopware 6.7

Shopware 6.7 introduces significant enhancements to the import/export functionality, providing developers with powerful tools to create custom data synchronization solutions. This blog post explores the technical aspects of building custom import/export profiles, focusing on the new capabilities and best practices for implementation.

Understanding Shopware 6.7 Import/Export Architecture#

The import/export system in Shopware 6.7 has been completely revamped to provide more flexibility and control over data processing. The core architecture now utilizes a plugin-based approach where custom profiles can be registered and executed programmatically.

Key Components#

The import/export functionality is built around several core components:

1. ImportExportProfile: The main profile entity that defines how data should be processed
2. ImportExportLog: Tracks all import/export operations for debugging and monitoring
3. DataConverter: Handles the transformation of data between formats
4. EntityReader/Writer: Manages the actual reading and writing of entities

Creating Custom Import/Export Profiles#

Step 1: Define Your Profile Structure#

To create a custom import/export profile, you need to extend the existing import/export service. The first step involves defining your profile configuration:

<?php declare(strict_types=1);

namespace MyPlugin\Profile;

use Shopware\Core\Framework\DataAbstractionLayer\Search\Criteria;
use Shopware\Core\Framework\DataAbstractionLayer\Search\Filter\EqualsFilter;
use Shopware\Core\Framework\Plugin\Context\InstallContext;
use Shopware\Core\Framework\Plugin\Context\UninstallContext;
use Shopware\Core\Framework\Plugin\Plugin;
use Shopware\Core\Framework\Util\Random;

class CustomImportExportProfile extends Plugin
{
    public function install(InstallContext $context): void
    {
        // Register custom import/export profile
        $this->registerCustomProfile($context);
    }

    private function registerCustomProfile(InstallContext $context): void
    {
        $profile = [
            'name' => 'Custom Product Import',
            'sourceEntity' => 'product',
            'destinationEntity' => 'product',
            'format' => 'csv',
            'mapping' => [
                ['source' => 'id', 'target' => 'id'],
                ['source' => 'name', 'target' => 'name'],
                ['source' => 'price', 'target' => 'price'],
                ['source' => 'categories', 'target' => 'categories'],
            ],
            'settings' => [
                'delimiter' => ';',
                'enclosure' => '"',
                'escape' => '\\',
            ]
        ];

        // Register profile in database
        $this->createProfile($profile);
    }
}

Step 2: Implement Data Conversion Logic#

The heart of any import/export profile lies in its data conversion capabilities. Shopware 6.7 introduces new interfaces for implementing custom converters:

<?php declare(strict_types=1);

namespace MyPlugin\Converter;

use Shopware\Core\Framework\DataAbstractionLayer\Search\Criteria;
use Shopware\Core\Framework\DataAbstractionLayer\Search\Filter\EqualsFilter;
use Shopware\Core\Framework\ImportExport\Convert\ConvertContext;
use Shopware\Core\Framework\ImportExport\Convert\ConverterInterface;
use Shopware\Core\Framework\ImportExport\Convert\ConverterRegistry;
use Shopware\Core\Framework\ImportExport\Exception\InvalidDataException;
use Shopware\Core\Framework\ImportExport\ImportExportLog;
use Shopware\Core\Framework\ImportExport\ImportExportProfile;

class CustomProductConverter implements ConverterInterface
{
    public function convert(array $data, ConvertContext $context): array
    {
        // Process and transform data according to your business logic
        $convertedData = [];
        
        foreach ($data as $row) {
            $convertedRow = [
                'id' => $row['id'] ?? null,
                'name' => $this->processProductName($row['name'] ?? ''),
                'price' => $this->processPrice($row['price'] ?? 0),
                'categories' => $this->processCategories($row['categories'] ?? ''),
                'custom_field' => $this->generateCustomField($row)
            ];
            
            $convertedData[] = $convertedRow;
        }
        
        return $convertedData;
    }

    private function processProductName(string $name): string
    {
        // Custom processing logic for product names
        return trim(htmlspecialchars_decode($name));
    }

    private function processPrice(float $price): array
    {
        // Process price according to your requirements
        return [
            'gross' => $price,
            'net' => $price / 1.2,
            'currencyId' => $this->getCurrencyId()
        ];
    }

    private function processCategories(string $categories): array
    {
        // Parse and convert category string to array
        return explode(',', $categories);
    }

    private function generateCustomField(array $data): string
    {
        // Generate custom field based on source data
        return md5(implode('|', $data));
    }
}

Step 3: Register Custom Converters#

To make your converters available within the import/export system, you need to register them properly:

<?php declare(strict_types=1);

namespace MyPlugin\Profile;

use Shopware\Core\Framework\DataAbstractionLayer\Search\Criteria;
use Shopware\Core\Framework\Plugin\Context\InstallContext;
use Shopware\Core\Framework\Plugin\Context\UninstallContext;
use Shopware\Core\Framework\Plugin\Plugin;
use Symfony\Component\DependencyInjection\ContainerBuilder;

class CustomImportExportProfile extends Plugin
{
    public function build(ContainerBuilder $container): void
    {
        parent::build($container);
        
        // Register custom converters
        $container->register('my_plugin.custom_product_converter', CustomProductConverter::class)
            ->addTag('shopware.import_export.converter', ['type' => 'product']);
    }

    public function install(InstallContext $context): void
    {
        parent::install($context);
        
        // Register the custom profile with the system
        $this->registerProfile($context);
    }
}

Advanced Configuration Options#

Custom Entity Mapping#

Shopware 6.7 allows for complex entity mapping through the use of custom mappers:

<?php declare(strict_types=1);

namespace MyPlugin\Mapper;

use Shopware\Core\Framework\ImportExport\Mapping\MapperInterface;
use Shopware\Core\Framework\ImportExport\Mapping\MappingContext;

class CustomEntityMapper implements MapperInterface
{
    public function map(array $data, MappingContext $context): array
    {
        // Custom mapping logic for complex entity relationships
        foreach ($data as &$row) {
            if (isset($row['product_number'])) {
                $row['custom_product_id'] = $this->generateProductId($row['product_number']);
            }
            
            if (isset($row['customer_group'])) {
                $row['customer_group_id'] = $this->resolveCustomerGroup($row['customer_group']);
            }
        }
        
        return $data;
    }

    private function generateProductId(string $number): string
    {
        // Generate custom product ID based on your logic
        return 'PROD-' . strtoupper($number);
    }

    private function resolveCustomerGroup(string $group): ?string
    {
        // Resolve customer group name to ID
        return match ($group) {
            'vip' => 'vip-group-id',
            'regular' => 'regular-group-id',
            default => null,
        };
    }
}

Error Handling and Logging#

Robust error handling is crucial for import/export operations. Shopware 6.7 provides enhanced logging capabilities:

<?php declare(strict_types=1);

namespace MyPlugin\ImportExport;

use Psr\Log\LoggerInterface;
use Shopware\Core\Framework\ImportExport\ImportExportLog;
use Shopware\Core\Framework\ImportExport\ImportExportProfile;
use Shopware\Core\Framework\ImportExport\Process\ImportProcess;

class CustomImportProcessor
{
    private LoggerInterface $logger;

    public function __construct(LoggerInterface $logger)
    {
        $this->logger = $logger;
    }

    public function processImport(array $data, ImportExportProfile $profile): array
    {
        $results = [];
        $errors = [];

        foreach ($data as $index => $row) {
            try {
                // Process individual row
                $processedRow = $this->processRow($row);
                $results[] = $processedRow;
                
                $this->logger->info("Processed row {$index}", [
                    'profile' => $profile->getName(),
                    'row_data' => $row
                ]);
            } catch (\Exception $e) {
                $errors[] = [
                    'row_index' => $index,
                    'error_message' => $e->getMessage(),
                    'error_code' => $e->getCode()
                ];
                
                $this->logger->error("Error processing row {$index}", [
                    'profile' => $profile->getName(),
                    'error' => $e->getMessage(),
                    'row_data' => $row
                ]);
            }
        }

        return [
            'results' => $results,
            'errors' => $errors,
            'total_rows' => count($data),
            'successful_rows' => count($results)
        ];
    }

    private function processRow(array $row): array
    {
        // Your row processing logic here
        return $row;
    }
}

Performance Optimization#

Batch Processing#

For large datasets, implementing batch processing is essential:

<?php declare(strict_types=1);

namespace MyPlugin\Batch;

class BatchProcessor
{
    private const BATCH_SIZE = 1000;

    public function processLargeDataSet(array $data): array
    {
        $results = [];
        $batches = array_chunk($data, self::BATCH_SIZE);
        
        foreach ($batches as $batchIndex => $batch) {
            $batchResults = $this->processBatch($batch);
            $results = array_merge($results, $batchResults);
            
            // Clear memory between batches
            gc_collect_cycles();
        }
        
        return $results;
    }

    private function processBatch(array $batch): array
    {
        // Process batch of data
        return array_map([$this, 'processRow'], $batch);
    }
}

Memory Management#

Proper memory management is crucial when dealing with large imports:

<?php declare(strict_types=1);

class MemoryOptimizedImporter
{
    public function importWithMemoryManagement(string $filePath): void
    {
        // Set memory limit for the process
        ini_set('memory_limit', '512M');
        
        $handle = fopen($filePath, 'r');
        if (!$handle) {
            throw new \RuntimeException('Could not open file: ' . $filePath);
        }

        // Process file line by line
        while (($line = fgets($handle)) !== false) {
            $this->processLine($line);
            
            // Clear memory periodically
            if (ftell($handle) % 10000 === 0) {
                gc_collect_cycles();
            }
        }

        fclose($handle);
    }
}

Integration with Existing Systems#

API Integration#

Custom import/export profiles can be integrated with external APIs:

<?php declare(strict_types=1);

namespace MyPlugin\ApiIntegration;

use GuzzleHttp\Client;
use Shopware\Core\Framework\ImportExport\ImportExportProfile;

class ApiBasedImporter
{
    private Client $httpClient;

    public function __construct(Client $httpClient)
    {
        $this->httpClient = $httpClient;
    }

    public function importFromApi(ImportExportProfile $profile, string $apiUrl): array
    {
        $response = $this->httpClient->get($apiUrl);
        $data = json_decode($response->getBody()->getContents(), true);
        
        // Process and transform API data
        return $this->transformApiData($data);
    }

    private function transformApiData(array $data): array
    {
        // Transform API response to Shopware compatible format
        $transformed = [];
        
        foreach ($data as $item) {
            $transformed[] = [
                'id' => $item['id'],
                'name' => $item['name'],
                'price' => $item['price'],
                'external_id' => $item['external_id']
            ];
        }
        
        return $transformed;
    }
}

Best Practices and Recommendations#

1. Validate Data Early: Always validate data before processing to prevent errors downstream
2. Use Transactions: Wrap import operations in database transactions for atomicity
3. Implement Proper Error Handling: Log errors comprehensively for debugging purposes
4. Monitor Performance: Track import/export performance and optimize accordingly
5. Handle Large Datasets: Implement batch processing for large data sets

Conclusion#

Shopware 6.7's enhanced import/export capabilities provide developers with unprecedented flexibility to create custom data synchronization solutions. By leveraging the new architecture, implementing custom converters, and following best practices, you can build robust import/export profiles that seamlessly integrate with your existing systems and business requirements.

The key to successful implementation lies in understanding the underlying architecture, properly handling data transformations, and ensuring optimal performance for large datasets. With these tools and techniques at your disposal, you can create sophisticated import/export solutions that meet even the most complex business requirements.