NavigationArts.com

our company

Contact Us

Navigation Arts Location Map

Connect

Find out more about our consulting services. McLean. Boston. New York.

Solutions

Thinking Big

Clark Construction

Digital Innovation

Reimagining how our clients can do business in the digital world.

Thinking Small

University Website Design

Mobile & Responsive

A responsive design approach engages users wherever they are. Web. Tablets. Mobile.

User Experience

Left Brain

PFC Energy

Critical Thinking

User-centered digital strategies help our clients surpass their competition.

Right Brain

National Aquarium

Immersive Design

Exciting and memorable visual experiences guide users through digital channels.

Technology

Sitecore

sitecore

Certified Partner

NavArts is one of only 7 Sitecore partners in the USA with 4 or more certifications.

Drupal

Drupal 7 Content Management

Open-Source

NavArts brings proven process and UX skills to the popular open-source platform.

our work

Case Study

State Department

A new site for the State Department guides journeys across the globe.

Design

Portfolio_Water

Portfolio

Cutting-edge design to revolutionize the world of user experience.

Insights

Webinar

Nicole Dingess

What is UX?

UX should be in your digital vocabulary, but do you fully understand its significance?

Search

Integrating External Applications: The Hidden Costs by:


January 11th, 2010

So you are at the beginning stages of your new content management system (CMS) implementation project.  You are finally going to get rid of the very painful, homegrown content management system you are currently using.  Your head is filled with dreams of eliminating all of your manual processes and replacing them with fully automated integration points between your shiny new CMS and your entire hodgepodge of legacy systems.

As you approach this implementation project it is important to carefully consider each integration point and really evaluate the potential tradeoffs to determine if an automated integration is necessary.  Automated integrations can carry a number of hidden costs.

The rise of XML, HTTP Web Services and Software as a Service (SaaS) has made communications between back end systems easier than ever.  Proprietary data formats are becoming a thing of the past.  Application Programming Interfaces are more and more available.  Also, modern CMSes are making it easier and easier to integrate custom code in to your site.  All of this leads to an increased desire for automating and integrating disparate systems.

A few simple examples can illustrate the issues involved.  First, let’s say that you would like to have a page on your new site that lists contact information for key staff members. You already have an internal, home-grown Microsoft Access database to manage staff information.  You have just acquired a new CMS built on top of Microsoft SQL Server. You think it would be great to automatically display that information from your internal system, with no manual intervention.  So it should be pretty easy – right?

On the surface, the integration seems very straightforward.  It wouldn’t take long to build a proof of concept showing that it is possible to move the data from the internal system to your CMS.  You will only be displaying data, so it will be a read-only data pull.  But as you look more closely, you must consider a number of questions:

  1. Can the CMS query the staff system directly?  Often, the web site lives in a completely different hosting facility than your back end systems, and the CMS cannot see the internal system.
  2. Can you set up secure communications between the CMS and the staff system?
  3. Are you comfortable poking holes in your firewall to support these communications?
  4. Does the home-grown system have some sort of public API or accessible database?
  5. If there is no way for the CMS to query the staff system directly, is there some way for the staff system to produce a snapshot file that can be imported in to the CMS?
  6. If you go with the file export method, how are you going to transport the file to the CMS server?
  7. Can you set up a secure FTP server in a way that protects privacy-protected information?
  8. Will you need to encrypt the file, or is the information not important enough to require encryption?
  9. What types of processes will you need to set up internally to kick off this export process?  Will it be nightly, weekly, monthly?
  10. How will you monitor this export process?  Who will get notified if the export fails?
  11. On the CMS import side how will you handle corrupt files or files that are not in the correct format?
  12. Who will be notified if the import fails?
  13. What type of data import architecture are you going to use?
  14. Are you going to actually import this data in to the CMS or are you simply going to write code that gets it from the staff system and display it on the web page?
  15. If you import it in to the CMS are you going to allow CMS authors to modify it?  If so, how are you going to handle differences between what’s in the CMS and what’s in the staff system?
  16. What if there are certain staff people that you don’t want to display on your web page?  Can you come up with some way to flag those people in the CMS so that they are not displayed?
  17. What if there is extra information that you would like to display on the web page that is not in the staff system – like a bio?  Can you come up with some way of adding information to a staff record in the CMS and still allow updates to come from the staff system?
  18. During development how are you going to test the integration?  Is there a testing version of the staff system available?  Usually you are going to want to create a development environment that replicates the production environment as much as possible.  You are going to want to be able to easily add, edit and delete staff people so that you can test the code.  You are also going to want to be able to simulate export and import errors so that you can handle them appropriately.
  19. If your solution is to not store any information in the CMS and to query the staff system every time the page is loaded, how are you going to handle disruptions in communication?  What will happen if the staff system is unavailable?  Also, how will this impact performance?

As you can see, a simple integration with a small back end system can generate a lot of questions and a lot of dependent functional and technical requirements.  Often times when people envision simple integrations like this they neglect to think through the possible consequences.  People like to imagine being on the “happy path,” where everything works correctly.

When faced with a set of questions like this it is time to really consider the options.  This leads to a different set of questions.

  1. How often does this data really change?  Your staff listings may only change very occasionally.  So even though automation sounds good, this may be a good candidate for a manual update.
  2. Is there some more simple way of doing the data transfer? Could a CMS author simply do a copy and paste of the information from one system to another?  Could a CMS author do a manual file transfer on as needed basis?
  3. Can we simplify the requirements to reduce the complexity?
  4. How important is it that the data be completely up to date at all times?
  5. How much of a burden will it be for CMS authors to update the data manually?  How likely are we to have copy and paste errors or other human errors if we do it this way?

Any integration is a trade off.  Usually the point of back end system integrations with your CMS is that you are willing to spend some time up front, developing and writing the code to handle it. That pays off in the long run through increased accuracy and decreased maintenance costs.  Sometimes these decisions are no brainers: if you are selling products, obviously you will integrate your website with your back-end product management database, because you would never enter all of your products in manually.

But sometimes the decision is less clear.  Is the requirements gathering, development, testing, monitoring and maintenance (and increased application complexity) worth the benefits gained from the integration?  It is important to understand everything that can go in to a system integration effort before you can accurately answer that question.

Frequently, the best solution is to use a phased approach.  Keep things simple initially.  Maybe the first iteration of your new CMS-powered web site will only include one or two key integration points.  If you can, defer the other suggested integrations to a later phase.  That way you can concentrate your time on the re-design and implementation of your website.  And you will be able to produce a new site quicker and show results sooner.  Reduced complexity will also lead to a more reliable first version of your site.  Over time you can add (or remove) integration points as you see fit.  You may find that a manual process just isn’t working, or you may find that a system integration can be replaced by a manual process.

Just make sure that you really take the time to fully investigate all of the hidden costs of what at first blush seems to be a “simple” data integration.

Tags: Web Development, , , ,


Corey Burnett

by Corey Burnett

Lead Web Application Developer

@coreyburnett



April 2014
M T W T F S S
« Mar    
 123456
78910111213
14151617181920
21222324252627
282930