Allow the job to continue when a read transform fails (2 replies)
Andy, thanks for this.
This sounds like a reasonable request. There are some Reader pipeline issues which need to be solved (there are 4 interfaced objects when reading data, which may require altering one or two of them), so it's not quick fix.
Probably won't make it for the next release, but possibly the one after.
Closed due to inactivity.
As per support request 983.
When a read transform fails, it should skip the error file and process any good files it finds. It should log the error to the audit log and if possible allow functionality for moving the file to another folder on error.
If all files are error files, then when it moves to the next transform there will be no data so there is no issue with data integrity here. I would expect most integrations to be developed on a file by file basis, integrations aren't likely to be split over multiple files so dropping the error file is a safe action to perform as it's isolated in that sense.
If there is a split file scenario, then provide an option to control this behaviour.