How did intelligent search evolve from enterprise search? New opportunity and foresight unlocked by smart insight engines.

What is intelligent search, how did it evolve from enterprise search, and what benefit do insight engines bring to your business? Find out why traditional search no longer works, how intelligent search empowers knowledge workers, and what the future holds for AI and insight engines.

How Netflix disrupted search to deliver data insight

Netflix collects masses of data from more than 200 million subscribers around the clock. Everything you watch. When you started and stopped watching it. Where you were when you watched it and what device you watched it on. Netflix watches you watching it – and this data tells Netflix what you most probably want to watch next. Netflix disrupted search. And it’s seriously insightful. That’s what makes it so popular and successful.

How did the Netflix streaming service get so experiential? Powered by automated AI, machine learning and algorithms, the tech team behind Netflix collate and enrich this recorded data to unlock valuable insights about your viewing preferences (and more). And these data insights are used to predict your future viewing intent.

The upshot. Netflix serves highly personalised viewing recommendations directly to your device. The benefit. You no longer have to waste time searching through reams of videos to find something to watch. That timesaving means you also get to enjoy more videos that directly match your preferences. And every time you watch a recommended video that meets your needs, the algorithm improves on a continual cycle to make the Netflix experience even more insightful and effortless for you as a user.

Serve or search – what’s the state of play at your workplace?

But how does this serve vs search scenario play out in your day-to-day work? For example, as a knowledge worker, how often are highly personalised ‘insightful’ recommendations served directly to your device so you can, say, predict market trends? Like many knowledge workers, if you rely on a traditional enterprise search engine, the user experience is far more laborious.

Instead of automated serve, you get manual search. And most of the time, you’re unlikely to find the exact information or answers you need, no matter how hard you ‘search’. When you do find it, unlike Netflix, you have to spend even more time trying to unlock insightful knowledge from it. With enterprise search, nothing is ‘served’, its seriously undiscerning, and that’s why it’s not so popular or successful.

The good news? Your workplace doesn’t have to be deal with data in this way. Netflix doesn’t own AI! Machine learning and algorithms are readily available to all enterprises choosing to adopt them. It’s also a matter of whether your business chooses to invest in powerful AI solutions now (like Netflix) or later (and risk flopping). Entrepreneurial businesses are always at the front of the queue. And, right now, they’re busy replacing clapped-out traditional enterprise search systems with significantly more powerful AI-steered intelligent search solutions.

Much like Netflix, intelligent search uses AI to log and harvest data (plus a whole lot more, which we’ll come to later). In brief, intelligent search can unlock valuable insights about your customers, clients, and market sectors. And these data insights can be used to inform better decision-making, resolve problems, and predict new business opportunities. The upshot. Highly personalised insights are ‘served’ directly to your device. The benefit. You no longer have to waste time ‘searching’ through reams of information to find the insight you need. That timesaving means you also get to spend more time putting these insights to work. And every time you select an insight that meets your needs, the algorithm improves on a continual cycle to make your user experience even more insightful and effortless.

But we’re getting ahead of ourselves. Before we delve deeper into how intelligent search lifts the lid on new opportunities and future outlooks, let’s reflect on how its predecessors – enterprise search engines and close relative web search engines – first came about. And what happened when they met their nemesis, AI.

Reflecting on search – why does search no longer work?

Back in the day, Industry 3.0 marked a new dawn for search. Generating vast reserves of data, automation and globalisation fuelled a modern Information Age ruled by ICT. Data became kingpin, driving enterprise growth. Proficient in leveraging technological solutions to reap value from this abundant data, knowledge workers needed to find information fast to predict market trends ahead of competitors. Answering this call of duty, web search and enterprise search flooded onto the market throughout the nineties and noughties.

Web search enabled knowledge workers to scour open internet systems for keywords or phrases, before switching up a notch to index information for more refined search results. Yahoo! delivered an attractive search engine interface that knowledge workers could scroll to find websites most aligned to their queries. Then Google marched to top spot with a faster and more personalised user experience, along with backlinked credibility.

As the world’s favourite web search, with 90% market share, Google still stands proud with its tribe of knowledge workers (Nooglers, Googlers, Xooglers) but, like all web search engines, it does exactly what it says on the tin. Renegaded to delivering search results that match manually inputted ‘search’ queries, web search falls short of delivering the intelligent insight that enterprises need for first-mover advantage.

Again, enterprise search enabled knowledge workers to sift through internal workplace systems, indexing structured and unstructured information stored across enterprise sources, repositories, and systems. Although easier to find corporate data, the sheer amount of information that today’s knowledge worker needs to manually wade - and its locations across varied systems - through makes enterprise search costly, cumbersome, and unscalable. And when it comes to the user experience, enterprise search lags behind web search. That’s not a great start.

“Data professionals are losing 50% of their time every week: 30% unsuccessful on data activities, plus 20% of their time building what already exists.”

Although fit-for-purpose at the time, enterprise search no longer works, it can’t be fixed, and the resultant cost for industry is huge. In fact, IDC’s State of Data Discovery and Cataloging report reveals that “The current state of data intelligence and knowledge is costing people time, and organizations money. Data professionals are losing 50% of their time every week: 30% unsuccessful on data activities, plus 20% of their time building what already exists”. This is estimated to cost each enterprise around $103 million in the US, and $49 million in Europe – a real deal-breaker for enterprise search.

New directions for intelligent search – how do insights open opportunity?

For knowledge workers to raise high-performance stakes in globalised economies packed with smart networks, autonomous factories, robotics, IoT, blockchain, cryptocurrencies, and the like, we need to give them a whole lot more.

Upsetting the apple cart, intelligent search and AI point progressive enterprises in a radically new direction. Powered by AI technology, intelligent search combines search results with valuable data insight to supply today’s knowledge workers with an altogether more automated and insightful user experience.

Personalised and intuitive, intelligent search not only finds relevant and interconnected information, but also discovers new and altogether more insightful answers. And then serves that information and insightful knowledge directly to knowledge workers, so they’re better equipped to make even more informed decisions.

Its new powerhouse, insight engines, discerns user intent. Or as Gartner puts it: “Insight engines differ from search engines in terms of capabilities that enable richer indexes, more complex queries, elaborated relevancy methods, and multiple touchpoints for the delivery of data (for machines) and information (for people).”

“Insight engines differ from search engines in terms of capabilities that enable richer indexes, more complex queries, elaborated relevancy methods, and multiple touchpoints.”

Insight engines don’t just place optimised information into the hands of knowledge workers. They serve rich insights directly to their table before they even check out the menu. Faster, smarter, and securer, insight engines don’t just search, index, and extract information. They also go the extra step to enrich and add meaningful context to every nugget of data stored across your enterprise. Knowledge workers can use this valuable data insight to analyse trends and patterns in data, form strategic and analytical decisions, and more.

A pressing need for increased knowledge worker speed also necessitates automation of routine analysis and information delivery. Insight engines meet this head on, becoming so integral that Gartner predicts 35% of knowledge workers globally will use insight engines to extract accurate and deep understanding from their diverse content sources by 2023. That’s a striking increase on just 15% in 2018.

On the back of this, Gartner predicts that by 2026, knowledge workers will halve the amount of time spent searching for information, because relevant information will be delivered straight to them within the context of their current tasks. Homing in on relevant and insightful data means insight engines also reduce information overwhelm, encouraging more focused knowledge work.

And although asynchronous and hybrid working (different tasks at different time points from different places) create new challenges for enterprises, accessing one automated insight engine means that all knowledge workers are working better, together. This minimises handovers and builds coherent business processes based on the same intelligent search solution. And when a knowledge worker leaves an enterprise, all that data insight stays put.

As our CEO, Steve Salvin, explains: “Organisations are looking for new, better ways to become data-driven and take control of their information. Insight engines offer a powerful solution to the challenge of discovering relevant, interconnected data and getting instant actionable insights precisely at the moment you need them. There’s huge potential in every organisation’s information, and an insight engine unlocks this.”

“Insight engines offer a powerful solution to the challenge of discovering relevant, interconnected data and getting instant actionable insights precisely at the moment you need them.”

MarketsandMarkets predict the global cognitive services market will be worth a staggering $15.28 billion by 2023, up from $2.59 billion in 2018. Intelligent search works, making insight engines one to watch. But exactly how does AI steer insight engines?

AI powers machine learning models and self-learning algorithms to better understand the user intent of knowledge workers. And then return accurate, relevant, and personalised insightful results. Fast. To do this, insight engines power through four key stages – discovery, enrichment, repository, and gateway – to create an enterprise-wide data mesh that interconnects all your newly enriched data to deliver new data insights directly to your knowledge workers. But we’ll delve deeper into how a data mesh drives enterprise-wide value for knowledge workers in our next blog.

Future possibilities for even smarter insight engines – where next?

On the cusp of Industry 5.0, anything is possible with early adopters, start-ups, and unicorns blazing a trail. Without doubt, smart gadgets, trackers, and sensors will continue to generate data from every facet of our lives. And forward-thinking enterprises will continue to unlock data insight to reach new heights.

Strong AI and even more powerful natural language processing has the potential to make intelligent search and insight engines even more personalised, customised, and collaborative, with smart glasses and wearables paving the path for visual and immersive user experiences. Voice-activated search and chatbots driven by complex machine learning could well be tools of trade for tomorrow’s knowledge workers. It’s conjecture whether deeper levels of consciousness can be developed between knowledge workers and ‘sentient machines,’ making them capable of performing intellectual tasks performed by humans.

AI and ground-breaking technology do have the potential to make our global supply chain more resource efficient – and our economy more sustainable, resilient, and circular. Applied for good, data and AI could also be used to promote more ethical development, remove bias, and mitigate harm. These are just some of the opportunities and outlooks AI could bring about.

At Aiimi, we will continue to invest in research and development to refine and advance our purpose-built intelligent search solution – the Aiimi Insight Engine – to help future-looking enterprises like yours tackle new challenges and predict changing trends ahead of Industry 5.0.

Read our latest eGuide – The Insight Opportunity: Uncovering the next generation of information intelligence – to see how intelligent search and insight engines can take you in a smarter direction.