Enterprise Search Pitfalls: Underestimating Implementation Efforts

I’ll say it: enterprise search is expensive and takes time to do properly.   The bigger the number of different repository types, the more complex your architecture is, the more you have in-house solutions or archaic systems in place, the more it will cost you time and money.  Seems obvious doesn’t it?  Then why do so many enterprise search projects go over budget?   Possibly due to the fact that enterprise search is often treated as an “add-on” to an existing infrastructure, or even as a necessary evil in some cases.   Many will treat search implementations like any other “simple” project.  To them the requirements gathering, vendor evaluation, and procurement process are the most daunting tasks.   Actual implementation and ongoing maintenance is too often neglected.

The reality is that many enterprise search implementations are far from “simple”.    Enterprise search often has large dependencies on many facets of your organization.  The complexity of an enterprise search solution often matches the complexity of all the systems it needs to interface with.   While many organizations end up maintaining different technologies in “silos”, search often forces these silos to come together.  Subject matter experts from several departments of your organization have to come together.  Your database team could need to explain the database structure and pin-point what tables and fields should be indexed, or even create new tables just for search.   Your CMS team could be called upon to optimize their publishing to ensure better meta-data quality.  Your network and security teams could be solicited to ensure enterprise search has the system access it needs, without compromising your environments overall security.  Getting everyone to do to their part at the right time is often challenging and the source of numerous delays.

Requirements are sometimes more volatile with enterprise search.  Change requests are frequent.  Some requirements are subjective in nature (e.g. what constitutes good relevancy).  Many decision makers are not familiar enough with enterprise search concepts or technologies, to properly formulate their requirements, or to cover all bases.  Finally, some implementation aspects of enterprise search are unpredictable.   For example, the data aggregation process is often one of the most consuming and unpredictable.  There are often marvelous discoveries being made by an organization as corporate data suddenly becomes searchable.  Are private documents now showing up publicly?  Why is legacy content appearing?  Why can’t I find a given document?  Many of these questions only appear when testing the search solution.  These questions can generate a lot of back and forth between business teams and search solution implementers, causing more delays.  This is especially true when enterprise search is suddenly used as a way to cleanup and structure corporate data repositories.

How to Prevent Underestimating

Be sure to approach enterprise search for what it is: a solution that can be quite complex to implement and takes time to put in place.

Take an iterative approach to development.  Proceed with small and frequent releases, targeting a limited subset of repositories at a time.  Do not try to do everything on first release; you’ll risk having to re-do parts of the implementation after testing occurs.  This advice can be applied to many IT projects, but it is especially true when re-indexing all your content can take a few days or weeks.

Come to the understanding in advance that an enterprise search implementation is ongoing.  Do not expect it to stop costing you money after your first production release.  Enterprise search should evolve over time, if only to adapt to content or technology changes in your organization.   You should budget for on-going enterprise search expenses every year.

When coming up with search requirements, be sure to list your “intent”, and not necessarily “how” functionality should be implemented.   Involve search experts in the requirements gathering process, making sure to give them the business information they need to figure out the best technical solution to meet your requirements.

Do your due diligence before starting a search implementation.  Make sure system accounts needed to access data sources and content repositories are created in advance, with proper access and privilege levels.  Ensure key personnel from all departments affected by enterprise search are readily available to answer any questions your search implementers might have.

Avoid the temptation to cut on testing and quality search expertise.  Having quality releases will save you money in the end.  Poor search implementations will likely have to be redone, or heavily modified several times over before being deemed acceptable.