Zero-Shot AI for Relational and Connected Data for Enterprises.

Current genAI shows a contractionary picture. Consumer AI produces widely acceptable content and praised. Whereas Enterprise AI projects fail at the rate of 95%. Only area where genAI is useful in coding applications that too has issues when codebase size grows. We need to understand why Consumer AI works and Enterprise AI. With all the 100s of billions poured into Large Language Model (LLM) AI now called generative AI, why promised changes in businesses not seen even after 3 years. 

Reason is simple, Enterprise data is mostly relational data, the data held in tabular column format stored in Relational databases, and simple formats such as spreadsheets hold a lot of data. This relational data is also held in new types of formats such as JSON stored in new type databases such as mongoDB collectively known as NoSQL databases. Relational databases using SQL language access the database. 

Enterprises need to extract relationships from relational data, even though data may be structured or unstructured. These relationships could show correlation or just how things are connected. For businesses it could be recommender systems where videos in Youtube or products in Amazon are shown or what is an alternative for a particular company. It could be deciding pricing based on several factors, finding related patents if one exists in the database. 

With this in mind, let us see how the LLMs are created. It was trained in large amounts of text data which is human created. That large language models mean exactly the same. It takes literally 100s of terabytes of text to create the pattern matching. Before the counter argument starts in your mind about videos and audio, yes video is mostly similar to text. Audio is text data just in different format. In order to understand the LLMs we need to discuss the structure of language. 

Languages have structure, that is why you can read this and understand. Most languages dont have exactly defined structure or what is commonly known as grammar. This grammar is a loosely defined structure. This means you cannot write rules and make the computer understand. That situation led to this current method of the Large Language Model. Because of a lot of examples needed to see the structure of language, AI needs a lot of data. And because of grammar's loosely defined structure, it is linear data rather than connected data. 

Languages come with few symbols, those include comma, astrohope, brackets, quotes. Just to coveny the meaning rather than defining or denoting any other relationship. All the meaning is conveyed sentence by sentence structure.  Human Languages with all its intricate grammar and variations are essentially a linear data structure. It conveys the meaning just by string of letters. It is complex too, the complexity comes from assigning various meanings to the same word. What is called a contextual understanding of language means that it needs huge amounts of data. 

In other words the complexity of languages is understood by analysing large amounts of data examples by AI.

Humans learn language from very few examples. We dont need to study every book in the world to master the language. With large and enough examples, every algorithm will work the same. i.e you dont need a special algorithm when you have large data. This works well when you are dealing with public data and you are asking questions about current events.

Both the positives and negatives of LLMs come from advantages and drawbacks of languages. LLMs work good and sometimes great with linear language type data. Fails when encountering Enterprise relational data. In other words, current generative AI or large language models are created for languages which are linear in nature. 

When LLMs are deployed for Enterprise applications, it will fail as it can't work with complex nested connected data. 

This leads to few options for Enterprises to use AI. AI is promising but can't work in the current way so companies can try a few things to make their data work with AI or try to implement AI in their work. AI works good comparatively in coding exactly because coding is similar to writing and programming languages are less complex than spoken languages. Other than that, Enterprises have few options. 

First option to transform the complex data into linear format or vectorize the data to feed into AI. This data transformation doesn't promise results but it can be the first step to do a pilot. Vectorization of data is good in some situations, it removes context in some situations. As the data is mostly relational, most of the genAI pilot fails in this stage itself. 

Second option is to finetune or distill the LLM model. This is customizing the LLM for a particular Enterprise needs. Finetuning is similar to calibration in other equipments where products with different specifications can be made. With LLMs, it could be adding more specific data, adding more parameters and so on. This is costly in terms of time and money, thus all companies can't afford it. Talent needed for this is also scarce, which puts another brake into project success. 

Third option is what if companies already have linear data and just want to search their data. They just wanted to search their textual data but the Enterprise search tools such as Solr/Elasticsearch can't bring relevent results. LLMs would be a good fit in that case. There too hallucination is the biggest issue. Hallucination denotes the behaviour of the LLMs to produce fake or non-existing facts. Fortunately there is an easy fix for that. Just verify the facts with data programmatically and if it passes the test, show the results. Else send that back to LLMs. However, simple as it sounds, this method also faces the challenge of how to differentiate between facts or results from LLM and how to make sure that search is accurate.  Remember the Enterprise search is an unsolved problem. This verification has a fancy name too: RAG Retrieval augmented generation. 

Fourth option is to use graph databases or building graph based data itself. This includes Knowledge graphs and Ontologies. Building a knowledge graph to make things fall into place automatically is a long held belief. Enterprises have treated building knowledge graphs as a long quest for extracting value. Cleaning up data, creating a graph based structure mostly a triplet based database is good. The question still remains, how to get value out of it? 

Graph databases suffer from the Curse of Graph Walking. Unlike relational databases where the database schema decides the query returns, in graph based structured data, there is no such a thing. There could be a lot of dead ends to graph nodes. Search timing increases exponentially with incremental increase in data. Reasoning attempts facing performance issues and glitches rendering the whole method as not useful 

Thus without a path breaking algorithm to solve the graph walking problem, data alone can't solve the problem. For Enterprise data, this is a delicate dance of algorithm and data. Data should exist in the correct format with efficient algorithms doing the work. What happens if either one or both doesn't exist?   

Introducing Zero-Shot AI for Enterprise relational data - NaturalText AI. 

NaturalText AI can enable Enterprises to work with Relational data with time and cost efficiency. The efficiency is the result of NaturalText efficient graph algorithms. It can work, both as a standalone AI or work with LLMs. Let us see the various solutions offered solving pain points in Enterprise data handling. 

First, Finding Relationships between data at scale (No limits for data size) - With graph based algorithms, NaturalText AI solves the problem of finding relationships. This could be finding which product customers bought togher, why customers are happy or angry about the a product, why customers are complaining in the feedback, building a list of lookalike companies from data, and so on. Wherever there is related data that needs to be extracted, NaturalText AI can easily solve it. All the Enterprise data in all format, text, RDBMS etc can be fed to NaturalText AI to see the relationships. This can weed out spam, NSFW and other anomalies in the data. In the same way, throw in your system logs, errors, anything that needs to be analysed for how to understand data is easy, fast and works well. 

Second, Zero-Shot Extracting Names from Data for building Ontologies, Classifications and Taxonomies.  NaturalText AI can easily extract names in fully Zero-Shot mode. No need for any pre-training or labelling. Hierarchical classifications can be built for all the data. Without any labelling, NaturalText AI can infer similar names from data. For example, it can automatically infer foot and meter belong to the same type of nouns just by inference. 

Third, Creating and Using Real Knowledge Graph for Enterprise wide data. Building a real knowledge graph that can reason, infer, deduce the conclusions from the data, would have a magical effect on Enterprises performance. This will be connected data, not just triplets. NaturalText's efficient graph algorithms solved the graph walking problem thus solving the fundamental issues in using knowledge graphs.  

Fourth, The magic of Zero-shot understanding of Unknown Unknowns. This is real heavy lifting of NaturalText AI. Just throw the data, NaturalText AI analyzes it automatically by creating contextual semantic clusters. This can work on any language, any size, any data. Filters out unwanted things such as spam, bias and NSFW. 

Fifth, Works in Any data format, any data representation. Data such as Chemical formulas including chemical finger-prints, Bio-sequences such as nucleotides can be analyzed and searched via NaturalText graph algorithms. This offers combined search, retrieval, reasoning and inference capabilities all under one platform and one database.  

Zero-Shot AI from NaturalText solves Enterprise pain points with relational data, offering unprecedented advantages.  

Struggling with Enterprise data, want to apply AI? Contact us at info@naturaltext.com