successfully integrated data with many Third Party Software
running on diverse platforms (different Microsoft Windows
platforms, AIX, HP-UX, Linux, etc..). Data Integration
has numerous demands that may not exist with a local
- moving large amounts of data over the network is
inefficient and unnecessarily consumes limited resources
versus batch processing - while some application's data
consumption can be processed via batch mode, this often is
not the case. In many cases, data must be provided in
real-time or near real-time basis and this puts a tremendous
burden on performance
processing - Large data chunks are cumbersome and often
requires additional resources to reprocess the data.
For example, repeatedly downloading the Who-Bought-What data
table is inefficient and consumes resources at the local
computer, the RISC box and network traffic. With incremental processing, ONLY the records that are
needed are downloaded.
- Some applications contain complex data structures that can
degrade performance of the integration software.
- Conflicts in data formats may result and it is important
to recast the data into the proper format. When that
is not possible, the integration software should be able to
handle this seamlessly, WITHOUT using manual intervention or
Data Complexity - It is
important to balance complex data integration with
performance. In many instances, third party software
demands data from many areas of the Prophet 21 system.
Efficiency must be paramount to extract the data so as NOT
to impeded the performance of the Third Party software,
integration software and on the RISC box.
Progress and VB combination:
By utilizing a
combination of Progress and VB, our efforts has yielded
significant performance gains that otherwise may not be
possible with a standalone Windows Application or as a
Windows Service. With this flexibility, we have been
able to provide a data source WITHOUT having to pull entire
tables or large chunks of data for reprocessing at the local
level. With a combination of Windows programming and
Progress Development, we are able to provide streamlined -
complete views of the data without performance degradation.
Some of our
approaches has been in:
Application and Progress
- Use the familiar Windows
Interface to create a front-end. This provides
terrific cost-performance benefits by reducing
production time and most importantly, reducing
the learning curve.
- Depending on the
requirements, we may employ a separate data
store such as Access, SQL Server or a MSDE
- We utilize the power of
Progress 4GL to provide the data processing to
retrieve the necessary data
- We transport the processed
data seamlessly to the Third Party Software
Service and Progress
- We create a windows service
that resides on a data server and is integrated
as part of its Windows Services
- Create the logic portion of
the integration utilizing Progress 4GL for
- Transport the processed data
back to the server
- Where possible, if a Window's
interface is not required, we prefer all or a
major portion of the integration program to
reside on the RISC box. This provides
significant performance gains.
- This method also simplifies
We customize the solution based on
requirements and utilize the most cost effective
approach to accomplish the goals.