Customer Forum

Syncing data using automated import jobs

Workbooks Support Posted: 2015-04-15 10:36
Two of the most powerful parts of Workbooks are the Import utility and the Process Engine. The Import tool lets you define mappings from a set of source data into multiple records in Workbooks, and to create or link records according to matching rules. The Process Engine lets you run scripts from time to time, for example on a regular schedule, and these scripts can fetch data from the outside world.
Bringing these facilities together means that you can build a solution to automatically import and update data from an external system into Workbooks, without needing to write large amounts of software to handle the complex bit: the mapping rules.
First you need a structured data source. It can be a set of files appearing on an FTP service (ftp or sftp) or a set of data held in a SQL database. You use this data and manually build an import job or jobs which you run to populate Workbooks; these jobs are your template.
Next you will create a scheduled process in Workbooks with the 'Requires External Access' checkbox ticked. From time to time the scheduled process fetches a set of data from the external system and uses the template import job or jobs to update Workbooks. The process of creating import jobs to run can include a step which removes source records which have already come across, so as to speed up the import job; ideally you won't need this step and your data fetch will only ever fetch records which have recently changed.
If you want to use this technique please get in contact with our Support team who can provide example process engine scripts to achieve this.