iWhat's New

Feature Preview in v4.0

The last 6 months has been busy! This lists the features of the forth-coming version of IMan, we count about 20 major, major enhancements & new features and the same again for the minor improvements.

Once we release we will be providing further detail as some features will need some review to ensure you are working post upgrade.

Upgrade Notice

Due to the scope of the changes the upgrade process for v4.0 is more involved than prior versions. We've tried to streamline this as much as possible by maintaining backwards compatibility.

You should familiarise yourself with the v4 upgrade process and notes document to note the possible breaking changes and to help plan your upgrades.

License Change Notice

Per previous email notices this upgrade changes the 'Unlimited' license.

Please consult the following notification for further information.

For test licenses please consult the upgrade notice.

Performance & Memory Usage

The major focus on this release is to improve performance and to reduce memory usage. The improvements have been made possible through an improved internal data structure, through data access improvements and through individual improvements throughout the entire application.

• Overall processing speed has been improved by between 20% to 2000% depending on data size, structure & transformations involved.
• Memory usage has been reduced by about 60-70%.

Aggregate Transform

Improved performance between 40% - 2000%, dependent on formulae and use.

Hierarchy Transform

The hierarchy transform has been rewritten to vastly improve speed and reduce memory usage.

Filter Transform

Improved performance between 40% - 200%, dependent on formulae and use.

Flatten Transform

The Flatten transform has been rewritten to vastly improve speed and reduce memory usage (see Flatten Transform Change Notice).

• A new option has been added to the flatten transform to specify the ‘join’ type of child records. You will have the option of using a strict inner join or an outer-type join (the default).
• When strict is selected the flatten operation will drop the parent (and children) records if one of the children levels have no children.
• The outer-join will simply populate empty children with empty values. This second option reduces the clunkiness in some integrations where complex transform logic was required in scenarios where there would be no children.

Map Transform

Improved performance between 25% - 250%, dependent on formulae.

Read Transforms

In addition to the performance & memory improvements across the product, read transforms are now multi-threaded resulting in faster processing and better utilisation of hardware.

Tasks (Email, File, FTP & Process)

Larger datasets will benefit from faster processing, sometimes by several orders of magnitude due to data access improvements.

Connectors

Performance improvements between 5% to 30%.

This is not the end!

We have committed to continue the performance and memory work made so far to all other areas of IMan. Over the next 6 months there will be a series of product updates to the remainder of the transforms: CSV, Excel & Fixed Width Writers; Aggregate, Filter & Map Transforms.

IMan Cookbook

We've released a new section in our documentation with short how-to's on topics which don't warrent a full training manual.

We're still developing some of the content, but this will be added to over the next couple of months and then gradually over time.

Lookups

Performance Improvements

The Lookup Function now has a Cache Results option. When enabled the function saves (or caches) the result of the function so subsequent calls to the lookup are checked prior to querying the database. Caching improves performance typically on larger datasets where the lookup function is requesting the same data. On a set of requests where the deviation of the requests resembles a standard bell curve, performance is improved by 50-65%.

The above is coupled with general performance improvements to the lookup function itself.

Existing ‘Lookup’ Function Improvements

The existing lookup function now allows you to:

• Specify a custom message used to report back to the user/audit report when the lookup doesn’t match any or matches more than one record.
• Return multiple fields from the Lookup function.
  • In prior versions if you needed to return multiple values from the same lookup with the same parameters multiple calls to the lookup function were required for each value you wished to return.
  • In the new version you can now pass an array of fields and the Lookup function will return an array of values.

New Lookupdb Function

Allows you to parameterise the database connection the Lookup function uses. This allows you to target different databases without having to have separate Lookup definitions.

New LookupWhere Function

Allows you to override the ‘where’ clause of the lookup function, by supplying a value to the function.

VbScript Transform Improvement

We have removed the debug messaging when a job is scheduled. This means when an error is raised in a transform or tasks using VBScript (Filter, Map, Aggregate, File Task), the audit report will look a lot cleaner. This makes it easier for end-users to decipher errors and to make corrections.

New FormatMessage Function

This new function allows you to construct a parameterised string.

Read Connectors

Read type Connectors (Amazon, Shopify, Magento, Salesforce) now allow you to define multiple Systems to query.

When multiple connectors are selected, each will be queried and be fed into a single IMan dataset.

This improves & extends the functionality the Amazon marketplace connector provided to each connector implementing a Read interface.

Database Writer

The Database (DB) Writer has been completely rewritten with the view to improve usability, extending functionality and improving data safety (see DB Writer Change Notice).

Usability Improvements

The ‘Map To Table’ text box has been replaced with an auto-complete drop down listing all the tables for the selected database. The following screenshot illustrates the drop-down against the Sage300 database.

The second other major improvement is the fields on the DB Writer no longer need to be the same as the column being inserted/updated.

We have now added the same type of drop-down to the allow you to select the column.

Auto-Increment & Sequence Column Handling

When inserting records with auto-increment or sequence type columns the DB Writer now allows you to write back to the IMan dataset the auto-generated value.

This function is database specific, since each database vendor implements this differently. As a consequence we’re not able to make this universal across all databases therefore we have implemented for: Access (yes, it’s still be used), SQL Server, MySql, Oracle & PostgreSQL.

Other DB Writer Improvements

• Increased data security and integrity.
  • The new version limits the ability for SQL injection attacks
• Now implements PROCESSED, INSERTED, UPDATED counters.
• Ability to insert or update binary type columns across all databases.
• Just a better experience!

Xml Writer

The XmlWriter continues to be improved, allowing easier generation of more complex documents.

As part of the release we will be improving/expanding the XmlWriter documentation to include examples on how to use templating, attribute paths, and other lesser used features.

Improved Templating

Templating reduces the work required to output complex Xml documents by providing a ‘template’ which is used by the XmlWriter to generate documents. Templating can be useful in cases where there are: numerous non-changing values; many attributes in a document; or where there are multiple namespaces defined.

The improvements made to templating make templating more intuitive to use.

Attribute Path Improvements

Attribute paths allow you to specify an attribute path in an XPath (see below) to make it easier to generate nodes where an attribute is defined.

/node1[@attribute=’value’]/node2

The changes build on from the templating improvements to cover more cases when these can be used in addition to a wrap up of several bug fixes.

Webservices

• Generalised support for TLS 1.2
• Support for Request Throttling.
• Basic Authentication allows you to specify additional header values.

Alert Only Report Group

• We have introduced an 'alert' only audit report group.
  This allows you to send emails to a specific report group whenever there is an error (terminating or non-terminating) in an integration.
  This new function is optional and works in conjunction with the existing audit reports.
• General clean-up of auditing controls - The auditing controls on the Options tab have now been moved Audit tab.

Email, FTP & SSH

• General compatibility for TLS 1.2 across tasks, transforms & functionality.

Connector Mapping & UI Improvements

• Improved performance when using the connector screen(s).
• Roll up of bug fixes and improved usability.

Sage Connector Improvements

SageX3

• Support for PU11.
• Add ability to write/import to a subprogram with a header/detail relationship.

Sage300

• Support for 2018.
• Add missing fields to various imports.
• Better handling of Date Optional fields
  • Date optional fields can now be specified in the localised form instead of the yyyyMMdd format needed in previous versions.
• I/C Transfer
• Can now update open transfers.
• O/E Order
  • Add Complete field on detail.
• P/O Order
  • Add Complete field on detail.
• RMA Entries
  • Can now update open RMA entries.
• Information Manager Notes
  • Added support to create & update notes.

Sage200

• Support for 2017 & 2018.

SageCRM

• Support for 2017 & 2018.
• Improved performance when using the connector.
• Added support to update multi-value selection type fields.

Marketplace Connector Improvements

Amazon Connector

• Ability to create shipments for Prime orders and the ability to print/export Amazon generated shipping labels for Prime orders.
• Add ‘IsPrime’ field to Order download to distinguish between Prime & regular orders.
• Ability to update available inventory from your ERP to Amazon.

CDiscount Connector Release

The connector will initially provide:

• Download of orders.
• Send shipping tracking details from your ERP to CDiscount.

eBay Connector

• Ability to complete and send shipment tracking data to eBay.