Slow API

Anyone who has had to get data into or out of Magento via its API will know about the dreadful performance. For transactional data (Orders, Shipments, etc) where the quantities are smaller this is usually ok, and with server infrastructure permitting, performance can usually be tuned to an acceptable point.

However, for master data such as product or customer records, where there are larger volumes and where there’s a need for frequent updates, the API is not only too slow, but can also result in database locking issues, update failures and the like.

To remedy this issue it is necessary to bulk update the data i.e. to execute the update server side, within Magento itself. Magento does have an import/export function, but is not conducive to a 24/7 eCommerce operation since it requires manual intervention and a specific CSV file format.

Our Magento connector addresses these by:

• Incorporating bulk/batch upload and download functionality for customer & product data. For example, bulk update functionality has reduced stock updates of around 2000 products from 3hours to under 5 minutes.
• Building the bulk functionality into the standard IMan integration flow, not only does this result in a configurable, easy to use mapping, but any errors are correctly reported onto the standard IMan audit report; enabling users to make any remedial action, easily and quickly.

Performance and memory usage

The poor performance of the Magento API is well known, but queries which are a result of API interactions can easily run afoul of memory issues (even the smallest ones). Settings to watch out for in your php setup are, memory_limit and max_execution_time and MySQL’s query buffer size innodb_buffer_pool_size. If you’re running into issues where API calls failing, altering these values may help.

We will be following up next month with a more comprehensive guide and tuning reference, so stay tuned.

Field Lengths

A common issue with any eCommerce/ERP integration are mismatches in field lengths; address fields are particularly sensitive. Any Magento implementation should review these, and how the fields will map to Sage, otherwise there’s a chance of miss-ships.

Fortunately, our connector has an inbuilt remedy to match address field lengths to the integrated Sage ERP solution.

Magento Extensions and their challenges to integration

One of the key benefits of Magento is the breadth and depth of the extensions (plugins) which can enhance Magento’s functionality in every perceivable way.

These extensions however can present a challenge to integration as a number of them will add and enrich data on any Magento object and this enriched data often needs to make its way into the ERP.

As Magento (pre version 2) lacks a descriptor or schema for its object model we can’t know ahead of time what fields are available. A means to dynamically interrogate the Magento objects to know what data is available so the necessary data can be passed through to your Sage accounting or ERP solution.

Again, our Magento connector does this transparently where product, order & customer attributes, and additional payment details are all automatically recognised, and the data can be mapped/integrated, based on requirements.

There are improvements to be made and our solution isn’t perfect. The existing Magento API doesn’t expose every piece of data, some of it squirreled away, with no API access. As part of our next release we’ll be implementing a feature which will allow lookups & translations directly against the Magento database (via a call to API), meaning every field can be gotten to.

Where do we go from here?

If you have any sticky issues or feature requests or just want to discuss the Magento connector in general please visit our forums:

/support/forum