STEPPED Xml Reader

The Xml data format allows you to query webservices over multiple 'stepped' requests. Stepping provides the ability to issue multiple requests to a service to build a single dataset. Each step passes to a lower transaction type a value or set of values, where the values can be used in the generation of a new request to build the lower level(s).

Stepped Xml Readers can only be used with HTTP IO Controllers. I.e. Stepped Xml Readers can only be used with webservices.

Stepped Readers have two slightly different modes of operations:

  • Seed List
    • The initial query made to a service provides a seed (or entry) list of values where the values are then used issues requests to obtain the data to make up the actual dataset. For example a service may be queried to provide a summary list of orders. Each summary record has a value such as an ID or URL which can be used to obtain the full actual order.
  • Parent to Child
    • Parent to Child is a more generalised version of the Seed List. Where the Seed List only provides a list of single values to the top-level transaction, any field from the parent record can be used in the request to obtain the child dataset. For example, in an eCommerce scenario a service returns orders with line items matching a specified criteria. However, a separate query may need to be made to obtain the payments made with each order.
The above modes are non-exclusive. They can be used together where the Seed List is used to generate the initial top-level dataset and the Parent to Child to generate each transaction level(s).

Url for Steppable Reader

The URL for each transaction type is specified in the Url for Steppable Reader field, and is the primary field for stepping. If the field is populated the Reader will step, otherwise if the field is empty the Reader will iterate the children as described in the Xml Reader. The value in the Url for Steppable Reader is parametrised with expando fields dependent on the type of stepping.

Seed List

The following sequence of steps are made when the Xml Reader is stepped from the initial request to the top most level:

  • An initial request is made to the webservice using the URL specified on the 'File Layout' tab.
  • The Xml Entry Path refers to the values from the response which will be used in the requests to build the dataset.
  • A new set of requests will be performed for each record returned from the initial request. The URL is generated from the value in the 'Url for Steppable Reader' for the topmost transaction, where the value for each record is referenced through the %SYS.ITEM expando value. Each record can refer either in full (i.e. a URL) or part (a value which can be used in a URL) to the record's URL.

The Reader will step only where the value in the 'Url for Steppable Reader' on the topmost transaction has a value. If this is empty no stepping will occur and the Reader will act without stepping.
  • The response of each request is then inserted into the IMan dataset. The Transaction XPath for the top transaction will be used as the entry point for the response. XPaths for the fields and child transactions are then relative to this path.

Example

URL

/admin/ordersList?fulfillment_status=unshipped

Initial Request Response

<orders>
  <order>
    <order_id>101</order_id>
    ...
   </order>
   <order>
     <order_id>103</order_id>
    ...
  </order>
  ...    
</orders>

Xml Entry Point

/orders/order/order_id

Url for Steppable Reader

/admin/order-%[SYS.ITEM]

Subsequent Requests

Request - /admin/order-101

Response

<order>
  <order_id>101</order_id>
  <customer>ABC001</customer>
  <address_1>123 Main Street</address_1>
  <item_lines>
    <line_item>
      <item_id>A1-103/0</item_id>
      <desc>NotePad</desc>
      <qty>5</qty>
      <price>12.10</price>
    </line_item>
    <line_item>
      <item_id>A1-401/1</item_id>
      <desc>Lamp</desc>
      <qty>1</qty>
      <price>45.99</price>
    </line_item>
  </line_items>
</order>

Request - /admin/order-103

Response

<order>
  <order_id>103</order_id>
  <customer>REL001</customer>
  <address_1>91 Brick Lane</address_1>
  <item_lines>
    <line_item>
      <item_id>C9-510/2</item_id>
      <desc>Read Trousers</desc>
      <qty>1</qty>
      <price>420.69</price>
    </line_item>
  </line_items>
</order>

tRANSACTION XpATH

/order (shown in bold).

Fields and Child Transactions

The XPath specified for the top transaction represents the entry point for the response. For example...

  • To reference the order_id property you would have a XPath of 'order_id'.
  • A child transaction consisting of the objects in the 'line_items' property array would have a Transaction XPath of 'line_items'.

Parent To Child

Stepping from a parent transaction to a child transaction works in very similar way to the seed list, except the step is happening from a parent to a child.

The Reader will step only where the 'Url for Steppable Reader' on the child transaction has a value. If this is empty no stepping will occur and the Reader will act without stepping.
  • If the 'Url for Steppable Reader' on a child transaction has a value, the Reader will 'step'. For each record in the parent a request will be performed. The URL is generated from the value in the 'Url for Steppable Reader' on the child transaction, where the values from the parent are referenced through the %FIELDNAME expando value, where FIELDNAME is the field id in the parent.
    This is in opposition to Seed List, where only a single value can be referenced through the SYS.ITEM expando value, Parent to Child allows you to reference multiple values by the field name.
  • The XPath should refer to the values from the response which will be used in the requests to build the dataset.
  • The response of each request is then inserted into the IMan dataset as a child to the parent. Similarly to the Seed List, the Transaction XPath for the child transaction will be used as the entry point for the response and the XPaths for the fields and child transactions are then relative to this path.