|Posted: 2018-11-12 09:18|
If you are using our SFTP/FTP Plugin, Stage 1 of this guide is completed automatically and can be skipped. See here for more information. If you are using a custom-built SFTP/FTP Process you should follow all steps of this guide.
This Script allows a Scheduled Process to be created which summarises the results of automated Imports created through an SFTP/FTP Process. This information can then be reported on and sent out in a Scheduled Email to the relevant people in your organisation to monitor the status of your imports.
Stage 1 - Create the Process
To create a Scheduled Process navigate to Start > Configuration > Automation > Processes > Scheduled Processes, once on this page select New Scheduled Process.
From here, there are several fields you will need to complete:
Click Save to reveal the Parameters and Logs tabs. On the parameters tab, there is one parameter that must be set for this process to function called "import_job_names". As the Import Job is created automatically, the SFTP/FTP script will create Import Jobs with a consistent name that is easy for this script to identify. This will need to be set to the names of any Import Jobs that you want this process to monitor for errors. If there are multiple Imports that you wish to monitor, separate each one with a new line.
Stage 2 - Configure the Report
We have created a Template Report for this Process to work with, however you may want to make some amendments to the Report Criteria to work with your specific requirements.
The Report Template can be found from Start > New > Report > Create a Report from a Template Report, then select “Template - API Data - SFTP Import Summary Report”.
This will load the Report shown below:
Criteria that you may wish to consider
Step 3 - Setup a Scheduled Email (optional)
Rather than having to periodically check the above Report to review Imports, you can create a Scheduled Email to send a summary of automated Imports to the relevant Users in your organisation. Details on how to setup and configure Scheduled Emails can be found on our Knowledge Base.
When setting up this Scheduled Email, consider how often you would like to be notified of any errors - this will need to work in line with your Report’s criteria and how often your processes are running. For example, if you have lots of processes that are dealing with specific files that run every hour, your report will need to take this into consideration. If you added a criteria that only shows Imports from the last hour, you could end up losing visibility of vital Import information. Lastly, we would also recommend only sending the Report when it is not empty, especially if you are running this frequently.
Step 4: Setup a Scheduled Bulk Action (optional)
A Scheduled Bulk Action can be configured to remove API Data records that are no longer of relevance automatically. This can be setup from the API Data Landing Page under Configuration > Automation > API Data. In the same manner as a Scheduled Email, we'd recommend considering how often, and how many SFTP/FTP scripts you have on your database. For example, you wouldn't want to delete API Data records that have not yet been set out in a Scheduled Email. For more information on setting up a Scheduled Bulk Action, see our Knowledge Base here.
All of these tools together will now provide you with a better method to monitor your SFTP/FTP Imports and any potential issues they may encounter.