STEPPED JSON Reader

The JSON data format allows you to query webservices over multiple 'stepped' requests. Stepping provides the ability to issue multiple requests to a service to build a single dataset. Each step passes to a lower transaction type a value or set of values, where the values can be used in the generation of a new request to build the lower level(s).

Stepped JSON Readers can only be used with HTTP IO Controllers. I.e. Stepped JSON Readers can only be used with webservices.

Stepped Readers have two slightly different modes of operations:

  • Seed List
    • The initial query made to a service provides a seed (or entry) list of values where the values are then used issues requests to obtain the data to make up the actual dataset. For example a service may be queried to provide a summary list of orders. Each summary record has a value such as an ID or URL which can be used to obtain the full actual order.
  • Parent to Child
    • Parent to Child is a more generalised version of the Seed List. Where the Seed List only provides a list of single values to the top-level transaction, any field from the parent record can be used in the request to obtain the child dataset. For example, in an eCommerce scenario a service returns orders with line items matching a specified criteria. However, a separate query may need to be made to obtain the payments made with each order.
The above modes are non-exclusive. They can be used together where the Seed List is used to generate the initial top-level dataset and the Parent to Child to generate each transaction level(s).

IO Controller path

The URL for each transaction type is specified in the IO Controller Path field, and is the primary field for stepping. If the field is populated the Reader will step, otherwise if the field is empty the Reader will iterate the children as described in the JSON Reader. The IO Controller Path is parametrised with expando fields dependent on the type of stepping.

Seed List

The following sequence of steps are made when the JSON Reader is stepped from the initial request to the top most level:

  • An initial request is made to the webservice using the URL specified on the 'File Layout' tab.
  • The JSON Entry Path refers to the values from the response which will be used in the requests to build the dataset.
  • A new set of requests will be performed for each record returned from the initial request. The URL is generated from the IO Controller Path specified on the topmost transaction, where the value for each record is referenced through the %SYS.ITEM expando value. Each record can refer either in full (i.e. a URL) or part (a value which can be used in a URL) to the record's URL.

The Reader will step only where the IO Controller Path on the topmost transaction has a value. If this is empty no stepping will occur and the Reader will act without stepping.
  • The response of each request is then inserted into the IMan dataset. The Transaction JSON Path for the top transaction will be used as the entry point for the response. JSON Paths for the fields and child transactions are then relative to this path.

Example

URL

/admin/ordersList?fulfillment_status=unshipped

Initial Request Response

{ 
  "orders" : [
    {"order_id" : "101", ... },
    {"order_id" : "103", ... },
    ...
   ]
}

JSON Entry Point

/orders/order_id

IO Controller Path

/admin/order-%[SYS.ITEM]

Subsequent Requests

Request - /admin/order-101

Response

{ 
  "order" :
    {"order_id" : "101", 
     "customer" : "ABC001",
     "address_1" : "123 Main Street",
     "item_lines" : [
       { "item_id" : "A1-103/0", "desc" : "NotePad", "qty" : 5, "price" : 12.10 },
	{ "item_id" : "A1-401/1", "desc" : "Lamp", "qty" : 1, "price" : 45.99 }
     ]
   }
}

Request - /admin/order-103

Response

{ 
  "order" :
    {"order_id" : "103", 
     "customer" : "REL001",
     "address_1" : "91 Brick Lane",
     "item_lines" : [
       { "item_id" : "A6-105/1", "desc" : "Trousers", "qty" : 3, "price" : 32.95 }
     ]
   }
}

tRANSACTION json pATH

/order (shown in bold).

Fields and Child Transactions

The JSON Path specified for the top transaction represents the entry point for the response. For example...

  • To reference the order_id property you would have a JSON Path of 'order_id'.
  • A child transaction consisting of the objects in the 'line_items' property array would have a Transaction JPath of 'line_items'.

Parent To Child

Stepping from a parent transaction to a child transaction works in very similar way to the seed list, except the step is happening from a parent to a child.

The Reader will step only where the IO Controller Path on the child transaction has a value. If this is empty no stepping will occur and the Reader will act without stepping.
  • If the IO Controller Path on a child transaction has a value, the Reader will 'step'. For each record in the parent a request will be performed. The URL is generated from the value in the IO Controller Path on the child transaction, where the values from the parent are referenced through the %FIELDNAME expando value, where FIELDNAME is the field id in the parent.
    This is in opposition to Seed List, where only a single value can be referenced through the SYS.ITEM expando value, Parent to Child allows you to reference multiple values by the field name.
  • The JSON Path should refer to the values from the response which will be used in the requests to build the dataset.
  • The response of each request is then inserted into the IMan dataset as a child to the parent. Similarly to the Seed List, the Transaction JSON Path for the child transaction will be used as the entry point for the response and the JSON Paths for the fields and child transactions are then relative to this path.