Migration to Shopware 6 for developers

Shopware Migration Assistant - Simple, flexible, extensible

To help merchants migrate their data as easily and intuitively as possible, Shopware provides the Migration Assistant. This wizard allows the user to easily transfer data from other shop systems to Shopware 6, if necessary, multiple times. Due to the function of multiple migrations, the Shopware 6 instance does not need to be reset when the data on the source system changes. The migration interface, like everything in Shopware, is easily extendable and adaptable. Thus, developers in agencies or plugin developers can add or change functionality that is necessary for their specific use case.

Concept and current profiles

The migration consists of several parts, which all depend on two superordinate components: Profiles and gateways. Profiles define the shop system that provides the data source. These profiles are, in turn, divided into individual gateways.

The gateways define in which way the migration data will be fetched from the source system. For example, in the Shopware 5 profiles, the migration uses an API provided by a plugin in the Shopware 5 source system, the Shopware Migration Connector. The Migration Connector plugin provides its own API routes tailored to the migration, in order to make the migration more efficient and flexible. The Migration Connector is designed in a way that allows other plugins to easily extend it.

One Gateway that is available in all Shopware profiles is the local gateway. It enables the migration of data from a local database. There are ready-made profiles for Shopware 5.4 and one profile for each minor version. For the migration of data from Magento 1, a profile for version 1.9 is currently available, the compatibility with lower minor versions is still being tested. For Magento 2, there are profiles for versions 2.0 to 2.3.

The general procedure for migration using the Shopware Migration Assistant starts with creating a connection. To do this, you choose a suitable connection name and define which profile and gateway to use. In the next step, the login credentials of the gateway must be entered, and a connection test is performed. If the connection test is successful, the data is now ready for migration. The data is bundled into several data selections (so-called DataSelections) since some of the data is interdependent, and therefore a single migration is not reasonable or even impossible. After one or more data selections have been activated, the migration is ready to start. It begins with a small data check, also known as pre-mapping. In this pre-mapping, the user must manually assign all data records that Shopware 6 cannot do automatically.

A good example of this is the order status. The available statuses usually differ between the source system and Shopware 6 and must be assigned manually. After all entries in the pre-mapping are set, the actual migration process can start.

In the first step of the migration, the data is retrieved from the source system via the gateway and converted into the corresponding Shopware 6 format using so-called converters. After the data import is complete, the data prepared by the converters is saved as a new Shopware 6 entity via the writers. If media need to be migrated, the next step is to initiate the media download or copy the media so that this process can run in the background due to the data size. When finished, the written data is indexed in the background so that all entities are searchable. If errors, warnings, or information occur within the migration, these are saved and can be viewed in the migration history overview.

Migration components

Gateways are used for communication between Shopware 6 and the source system and contain everything required to establish a connection and retrieve data. The following are the most important components of the migration and the general structure of the implementation. Profiles only serve to differentiate or identify which subcomponents, such as converters, are called. Thus, profiles do not contain any logic, but only functions for returning information such as name or version, which are then displayed in the administration.

Gateways are used for communication between Shopware 6 and the source system and therefore contain everything needed to establish a connection and retrieve data.

The gateway uses readers to retrieve data. The readers are identified via the registry and then called. A LocalReader, for example, executes SQL queries via the database connection, collects the returned data and makes it available to the gateway. The readers are selected from the registry using the support function, taking into account the profile, the gateway and the entity that is currently read. The actual retrieval of the data is then done in the fetchData method, which can be very complex depending on the entity.

DataSelections are used to fill the data selection in the administration of Shopware 6 and to enable the identification of the following components. Since these are only used for display and identification, no logic is implemented here. Only a structure is returned using the getData function. This structure contains, among other things, an identifier, all entities to be migrated, and a snippet name for display in the administration.

DataSelections contain multiple entities. There are DataSets to control the granularity. They identify the entity objects for migration so that other components, such as converters, only need to process one entity object.

For some entities, it is necessary to create another reader, the PremappingReader, in addition to the reader for the gateway. This reader provides a suitable selection of source and target options so that the user gets an overview of all manual assignments that are still required when starting the migration. The PremappingReader is dependent on the profile and data selection and must, therefore, implement a support function. In the getPremapping function, the source data and the target data for the pre-mapping are read and then returned. If a preselection is possible, or a selection already exists from a previous migration, this pre-mapping can also be implemented here.

The read data is then output by the gateway and forwarded by the corresponding service to the associated converter. The converter contains the actual logic of the migration and converts the incoming data into the Shopware 6 structure. The converter also uses a support function to identify the correct profile and the corresponding entity. Via the convert function, the converter receives the original data, converts it into a Shopware 6 structure, and returns it so that the converted structure can be temporarily stored in the database.

After the Shopware 6 structure has been created, the actual Shopware 6 entities are created. This is done by the writers. Writers contain a support function for identification so that the correct entity is written. In the writeData function, small adjustments or calculations are made if necessary, but usually, only the EntityWriter from the core is used to create the entity. As shown in the code example, the upsert function is used here, which creates a new data set or updates an existing data set, making multiple migrations possible.

The last component is the MediaProcessor, which must be implemented differently depending on the profile and gateway. For example, when migrating via the API, the media must be downloaded, and when migrating from a local database where the media also exists locally, the media must be copied in the server's file system. To make this possible, the MediaProcessor provides a support function that identifies the processor. The actual logic of media migration is implemented in the process function.

Extensibility

The Migration Assistant is designed as a simple and highly expandable system. For example, a uniform converter base per entity was developed for the migration of Magento based on Magento 1.9. Any converter that differs from this base can extend from the base converter and adapt the corresponding logic. This is how the converters of the Magento 2 migration were realized because a lot of things in Magento remained the same and only minor adjustments were necessary. This also applies to the readers of the gateways and pre-mappers. They have a basic logic that can always be extended. If an adaptation dependent on a minor version has to be realized, individual converters are provided for each minor version, so that a manual version change is not necessary. If the logic of a converter has to be adapted or extended, the decorator pattern is used. Assume the logic of the product converter of Shopware 5.5 needs adjustments, so that the manufacturers are selected by pre-mapping. To achieve this, the corresponding converter must be decorated and extended with its own logic.

Using the MappingService, the selected value can then be retrieved from the pre-mapping and added to the convertible data. For plugins with custom tables, it does not help to adopt an existing converter, since its entity and data would not be read. In those cases, the migration wizard can be extended with a separate data selection, which will read the data during migration. One of our well-known example plugins, SwagBundle, has a custom table and could extend the migration to migrate its own data as well. For this purpose, a separate DataSet would have to be created in the first step, then all further components can work with this entity.

In the next step, this DataSet would be added to a DataSelection, so this entity will be queried during the migration. You can create your own DataSelection or decorate an existing one.

If a DataSet is contained in one of the DataSelections, it is taken into account during migration and an attempt is made to call a suitable reader via the gateway. This reader must read the data from the table, put it into the correct structure if necessary, or enrich it with additional data before returning it.

After reading the data, the main part of the migration follows the conversion. A converter receives the data queried by the reader and converts it into the new data structure.

The last step is to write the converted data using a suitable writer. Since the predefined AbstractWriter already contains most of the logic, the definition of custom writers is very lean.

With the example described above, the data of the bundles would be migrated together in relation to the products. As you can see, it takes only a few steps to migrate the old plugin data to Shopware 6. But what to do if none of the existing profiles fit the source system? If, for example, an older Magento or Shopware version or even a completely different shop system is the source system and no profile exists for it, a separate profile must be written. The Migration Assistant is designed for this, so it is not necessary to create a custom implementation. A good example of this is the Magento profile. All Magento profiles have been implemented separately as a Magento plugin, but use the available logic of the Migration Assistant. In order to provide your own profile, you will need your own profile class in the first step. It does not include any logic, just a few properties with associated getters.

Next, a separate gateway is required, which can be implemented, for example, like the local database gateway of the shopware profile, so that access to the database is performed. Of course, other communication paths (e.g. REST-API, SOAP-API, etc.) can also be used in the own gateway, as long as the source system provides them. In the support function of the gateway, you must check your own profile so that your own gateway is also used by the migration.

Next, all entities of the source system that are to be migrated must be checked and the migration components for them must be created. As explained above, each entity requires a DataSet, a reader for the gateway, a converter to convert the data and a writer to store the Shopware 6 structure. The individual DataSets of the entities are then collected into different DataSelections, so that the entities in the data selection are available in the administration of Shopware 6. You can also implement custom pre-mapping readers if data must be mapped manually by the user.

Further information and HowTos can be found in the Shopware 6 documentation:

HowTos:

Documentation:

Conclusion

As you can see, the individual components of the migration can be extended and changed in many ways. Everything is prepared for the migration of your plugins or the migration from other source systems. Since Shopware wants to improve every day, here is a request: Whether you are a plugin developer or a Shopware enthusiast, please test the migration and extend it with your own customizations and profiles. Give us feedback about your experiences and give us suggestions for improvement. Your feedback will not only make the migration a little better but Shopware 6 as a whole.