Spreadsheet Excel Data - Create connections/links, Find relationships, Rank rows.

 Businesses still run on Spreadsheets. People find it easy to put the data and look at the data. Business executives spend time analyzing data manually or run queries to generate reports. That works for financial or numerical data. 

What happens when the data is mixed or mostly text such as databases? Manual analysis was the only way. Yes it was the only way. AI aka LLMs tried to change that by generating summaries or answering questions. Does that solve the needs of the businesses? No. Businesses need to understand the data and make sense of it. That gap still was there. 

NaturalText fills that gap with creating connections/links, groups, finding relationships and finally ranking the rows based on connections based popularity. 

Look at the image to understand the initial and final output. 

Let us see an example of a spreadsheet with 10,000 rows and 1000 columns. Some of the columns are for numerical data, some of the columns contain text data. It has both dense and sparse data, i.e. some rows contain data for most of the columns, some rows don't have data for most of the columns. 

This spreadsheet would be a perfect example of headache to the analyst. It has all the edge cases. Mixed data, sparse and dense data. So how to analyse this? None of the tools or AI in the market can analyze this. Because all the tools are designed for a specific format of data and solving a specific problem. 

NaturalText AI with Symbolic Zero-Shot AI solves this using graph based mapping. Each cell is mapped in a graph node and relationships are mapped in the graph space. This would be equivalent to comparing each row with every other row. This will create the groups and find relationships. 

Take a look at the below image. 

In this, some rows are connected with other rows by some columns but with another set of rows by different set of columns. This can be hierarchical too, something like a 3D map of connections and relationships where both vertical and horizontal connections are created.  ​In this

Row 10 connected to Row 40 by Cells in Col 13 and Col 17

Row 10 connected to Row 11 by Cells in Col 13 and Col 18

Row 10 connected to Row 57 by Cells in Col 10 and Col 11

Row 57 connected to Row 76 by Cells in Col 8 and Col 22. 

Row 57 connected to Row 1002 by Cells in Col 41 and Col 83

This multi mapping is the result of AI making sense of data via graph analysis.

No other method or AI can do this currently.  Number of cells connecting each row would be the ranking score or popularity. 

If more cells are connected to multiple rows that will make that row more popular. However, more similar rows will get the same score thus removing the possibility of spamming or manipulating the data. 

​Bias, Spam, NSFW all can be removed easily, only the genuine or good data would be taken for analysis. 

Business managers can verify that before sending the analysis out. 

Analysts, Business executives and CxOs can just see the map and understand what needs to be done. No need to depend upon the AI to make the summaries and keep asking different questions to the AI. 

Entire map of data is visible on the screen.  That visibility ensures the decisions are made fast, accurate and have the human touch. 

Adding this to Legacy Search, RAG + LLMs or NaturalText concept search. 

Once these connections are made, then these connections list or number of cells connected to each row used to create popularity rank. These ranking scores are fed to legacy search engines such as Solr or ElasticSearch for better ranking. For RAG + LLMs setup which cant handle the tabular connected data, this alternative can be used to enhance the results. 

NaturalText offers concept search as a replacement to both summaries and keyword based enterprise search. You can search in a document and across documents based on the connected concepts and popularity ranking between those rows or documents. 

This method can be extended to create ontologies and knowledge graph.  I will discuss that in a separate blog post. 

Looking for a tool to understand your spreadsheet data, reach us at info@naturaltext.com