KMWorld 2016- Enterprise Search & Discovery

Somewhere between the White House and the Trump International Hotel, between the anti-Trump, and anti-pipeline protests, there was another peaceful gathering in Washington D.C. last week… KM World 2016!  

This was the 20th anniversary of the event.  Norconex attended the Enterprise Search & Discovery stream, and it was obvious that the event has matured from the 20 years of experience with quality information sessions and vendor participation.

In talking Search, it was mentioned in several sessions that Search users want their Search to “work like Google”.  With Google employing tens of thousands of Search dedicated employees and the average company having less than one person dedicated to the same, it is no wonder that sometimes end users are left using a product that doesn’t fully meet their expectations.


The White House


Trump International Hotel














In many cases users are abandoning their Search application altogether to manually look for the content they need. This can cost a company in reduced productivity and in the case of online retailers, lost revenues.  But there’s hope!  With advancing technologies, dedicated vendors and Service Providers to work with, any company no matter the size can deploy a solution that works with their needs.

Some of the key areas of discussion I’d like to touch on in this article are Open Source, Machine Learning, the Cloud, User Interface, and Analytics.


Open Source continues to expand and is more and more widely accepted as a viable option for organizations of every size.  This can be to save costs on licensing fees, but also to provide more flexibility in how your Search is developed.   In some cases open source Search is being built alongside other products that include Search functionality (like Sharepoint) to enhance the Search experience beyond their standard offering.


Machine Learning has also come a long way, and a few vendors were on-hand to show off their products.  I was impressed with one product demonstration on how the Search results were displayed in an easily viewable chart format rather than a list.  However, it was said at the event that statistics are showing only 60-70% accuracy for these tools.   It was also said these products need very high query levels to reach the higher end of accuracy.  This means only the Search applications with thousands and millions of hits are getting full advantage of Artificial Intelligence today.  Assuming 60-70% relevancy is not enough, you will likely need some good old-fashioned human intervention to get the results to meet your expectations.

Also, if your organization is indexing all content, you may want to rethink this strategy and look at your content to determine what actually requires indexing.  It was said  that 60% of business data is not business data at all, but things like invitations to golf tournaments, pictures from the annual holiday party, duplicate documents or general user content such as personal emails that likely do not need to be included in your Search.  A Content Analytics tool can help you narrow down what content needs to be indexed to help with the relevancy of Search returns.


Another hot topic was moving your data and Search application to the Cloud.  The fear with moving to the Cloud had always been if your data will be secure.  Much like open source, organizations of every size are now embracing a move to the Cloud.  Many smaller companies who have limited IT resources are realizing that the big Cloud providers have security teams in place that can help their content actually be more secure than if they host on premise.   

The newer challenge around the Cloud is for multi-national organizations who have data in countries where data privacy laws are in place such as Europe’s Safe Harbour and more recently Russia’s data protection laws.  These legislations can regulate privacy, where their data can be stored, and also how and if that data can travel outside of the country.  Multinationals need to find a strategy to work with these laws potentially piecing together various Cloud providers with data centres in the countries in question, or doing a hybrid of Cloud and on premise.  


Once you’ve built out your Search infrastructure, what your end users see is the User Interface and the results that are displayed for their queries.  Rather than having a “Search Page” more and more companies are integrating the Search UI into their core user applications so the users don’t have to “search for the Search”.  

If you are going to include a user feedback option, best participation was recorded when the feedback was put near the Search UI,  but you will often get limited responses.   This is where Search analytics comes into play… taking user feedback (if available) along with information from your Search users behaviours to keep a pulse on how Search is performing and if your users are finding the content they were looking for.  A good Search Analytics product can help you to organize your Search data in a dashboard view, and provide an overall health-check to give you quick insights into where your Search is working, and where it needs some intervention to keep your Search running at an optimal level.

Regardless of whether you implement Search in-house or hire a team of experts, with all of the advancement in Search technology, you can put together all of the right pieces to provide a great Search tool for your employees and customers.

Valerie has over 20 years of experience working with government, SMB and enterprise level accounts to assist them in their IT and Telecom needs. As Vice President of Business Development at Norconex, she is passionate about helping businesses identify and implement their perfect search solutions.