Extract data

The Basics

Run shopify_transporter extract in order to extract data from your third-party platform.

Currently, Shopify Transporter supports extracting the following datatypes from the Magento 1.x platform:

• Customers
• Orders
• Products

This command has three required parameters:

1. object: The object you are trying to import: customer, order, or product.
2. first_id: The first id for the range of objects you are trying to extract.
3. last_id: The last id for the range of objects you are trying to extract.

There are also two optional parameters:

4. batch_size: The number of records that are retrieved from the source platform at one time. Use a smaller batch size if you are running into performance issues extracting your data. Defaults to 1000.
5. config: The configuration file to be used. Defaults to config.yml.

Configuration

First, if you are extracting data from Magento 1.x, ensure your Magento SOAP API has WS-I compliance mode set to "No". Configure this by navigating in the Magento admin to System -> Configuration -> Magento Core API -> WS-I Compliance.

Next, in order to extract data, API credentials must be provided in the configuration file. For example, to use the built-in Magento extractors, you must enter the following fields:

extract_configuration:
  soap:
    hostname: your-magento-host.com
    username: your-soap-username
    api_key: your-soap-api-key

To extract products, database credentials are also required in addition to the SOAP credentials. This is because it is not possible to extract complete product information using SOAP alone. Here is a sample of the keys required to extract products:

extract_configuration:
  soap:
    hostname: your-magento-host.com
    username: your-soap-username
    api_key: your-soap-api-key
  database:
    host: 123.456.88.99
    port: 3306
    user: username
    password: password
    database: magento_database_name

Sample Usage

The following command extracts customers with IDs in the range of 0..10000 from Magento:

shopify_transporter extract --object=customer --first_id=0 --last_id=10000 > magento_customers.json

The command outputs the extracted objects to STDOUT, which can be redirected using the > symbol to save the output to a file. In this example, the extracted customer objects are saved to a file called magento_customers.json.

Extracting products from Magento 1.x

To extract full product data from Magento 1.x, shopify_transporter extracts the following database tables and stores them in a /cache/magento_db folder in your project:

• catalog_product_relation
• catalog_product_super_attribute
• catalog_product_super_attribute_label
• eav_attribute_option_value

If there are newer products, delete the /cache/magento_db folder to ensure they are fetched again.

An alternative to providing database credentials to the gem is to manually extract these tables as CSV files and place them in the /cache/magento_db folder with the same names as the tables:

• catalog_product_relation.csv
• catalog_product_super_attribute.csv
• catalog_product_super_attribute_label.csv
• eav_attribute_option_value.csv

The names of the columns for these CSV files are expected to be the same as the names of the database table columns.