Refresh Data

Data Refresh Options

Keeping your data up-to-date

Omniscope files often contain data imported from files or databases being changed regularly. Typical data sources for Omniscope files include reporting views/tables regularly drawn from an often-changing database, or an Excel spreadsheet regularly provided (with changes) by others.

Omniscope remembers the location of the source of the external data imported, be it an Excel .XLS or other spreadsheet .CSV file, a database view saved as a .CSV or .TSV file, one (ODBC) or more (JDBC) database tables, or a folder of files or images on your PC. The external data source is linked to the Omniscope file and is referred to as a 'linked data source'. Maintaining this link enables Omniscope to automate the process of refreshing the data in .IOK files whenever the source data is changed. Moving the source file location relative to the Omniscope file will break the link. You can delete or restore links to external data sources at any time.

Omniscope .IOK files include a highly-compressed snapshot of all the data already imported. Access to the linked data source is NOT needed for you or others to view and explore data already in Omniscope. You can send an .IOK file to a friend who does not have access to the original data files, and they will still be able to explore the full copy of the data in the Omniscope file. Depending on which Edition of Omniscope you are using, your friend's .IOK file can refresh itself on opening, or even continuously (see Automatic refresh below) from a templated Source .IOK file accessible via the web or internal network. However, only your (publisher)  Source Omniscope file and your installation with access to the linked data source(s) can refresh the shared Source .IOK file data from the original data.

Refreshing data from linked source data files

{Omniscope Professional & Enterprise only} If the data source linked to the .IOK file is another data file, such as a .CSV or Excel .XLS file, when opened the .IOK file will automatically detect that its data source has been changed and will ask you if you want to refresh from source before opening. If you have licensed a Commercial Publishing version, changes to the source file can be automatically pushed out to users of the free Viewer over the web.

If Omniscope is unable to automatically detect changes to the external linked data source (if it's a database table being accessed via ODBC or JDBC protocols, for example), you can manually refresh on opening by choosing File > Import into current file > Refresh from source.  If you have added Bloomberg fields to your data set, and have access to Bloomberg on your PC, you can use File > Import into current file > Refresh from Bloomberg.

Warning: Omniscope does not currently support partial refresh of data tables. This means that columns which are not contained in the external data source are overwritten or lost if the Omniscope file is refreshed from a linked source (except for Formula columns, which are re-calculated). If you have added data columns from other sources, or maintain a lot of local data in the working Omniscope file, i.e. you have columns of notes, or pasted copies of e-mail correspondence, or references to pictures or map coordinate references which are not in your external data source, you must divide your working file into two separate Omniscope files prior to refreshing the columns originating from the linked source. This is easy to do. Once you have refreshed the Omniscope file containing columns from the external source, you can join both parts of the working file back together again. For more information on separating and joining .IOK files, see the  sections on the Export Data Wizard and the Import/Merge Data Wizard.

Refreshing from .IOK/.IOM linked source data files

In addition to other supported data files, it is possible to configure Omniscope .IOK/.IOM files which have other .IOK/.IOM files as linked data sources. This option has the advantage that only the data is delivered to the User Omniscopes, without changing the configuration of Views, Filters and Report Pages to match the opening configuration of the Source file, as happens when auto-refresh is used (see below).

Automatic refresh from live data sources (new in version 2.3)

Starting with version 2.3, you can now arrange for any number of desktop Omniscope Professionals to automatically refresh the data they display from a Source .IOK file running elsewhere, over your local network or across the web. If the Source .IOK file is in turn refreshing itself directly from the original source database view(s)/table(s) as these change, the combined configuration amounts to a scalable, near real- time live data refresh solution. If the Users are working, refresh awaits a pause in their activity. Each time Users re-open their copy of the auto-refresh linked file, it returns to the default opening configuration of the Source file.  To allow Users to maintain their own individualised configurations and import only the latest data, use the linked .IOK data source file option described above. For more information, see the Knowledge Base sections on Integration and Auto-Refresh.

I can't refresh from source!

If your linked data source is inaccessible, Omniscope will be unable to refresh from source. If you imported from a source file such as a .CSV or Excel .XLS spreadsheet, the linked source file must remain in the same relative location compared to the .IOK file. If you have moved either file independently, you can re-establish (locate) the source file by choosing Update linked source from the File > Import menu. If you imported directly from a database, check that the database server is started, and is accessible across the network if it is running on a different PC or server.

Editing & exporting data to linked files- be careful!

Omniscope allows you to edit cells in the Table View, right clicking on the cell to edit cell data, and right-clicking row headers to add/remove records (rows). You can also add and remove fields (columns) using the Edit > Manage Fields.

When you have finished, save your work in the IOK file as usual by choosing File > Save. If you originally imported from a linked data file (such as a .CSV or .XLS file), you will be asked if you also want to save back to source - i.e. to save your changes back to the linked source file. At any time, you can also manually save your edits back to the source file by choosing File > Export > Export back to source. (Professional & Enterprise Editions only}

Warning:  Whenever you choose to save back to source the entire source data file will be completely overwritten with your latest edits from the the Omniscope file. If you have deleted fields, these will also be deleted in the source file (therefore usually better to just hide them). If your source file is an Excel XLS file, all formatting and formulae will be deleted as the export is only for cell data. If you have a multi-tab spreadsheet, all the other tabs will be lost.

Omniscope does not currently support saving changed data sets directly into to database tables. If you need to save changes back to a database, collect your changed records in the Basket, then export a .CSV or .XML file containing the changed records. You or your database administrator can then use a database utility to import your changes directly into the database table(s). For more detail, see Using the Basket.

Unlinking/Re-linking to source data files

At any time, you can deliberately remove the persistent link between an .IOK file and its linked data source by choosing Unlink from source from the File > Export menu, or by unticking the Linked to source check box when saving the .IOK file using the File > Save as dialog.

Also, any time you are prompted to refresh from source or save back to source, there is always the option to click Unlink or Skip to prevent overwriting the source file. You can also overwrite the linked source file at any time by choosing Update linked source from the File > Export menu.

 

Next section: Useful tools

User Guide Top