Companies around the world generate huge amounts of data (80% of the world’s data is in business), but they struggle to extract insights from it. In many cases, this is because they are still trying to process the data using outdated technologies. Graphing technology offers a solution and is poised for explosive growth in the coming years, Gartner predicts that by 2025 graphing technologies will be used in 80% of data and analytics technologies.
Graphing technologies work by organizing unstructured datasets as linked structures of nodes and edges, and processing the data using algorithms and machine learning techniques, specializing in linked data structures. Despite their strength, two issues currently limit the widespread adoption of graphics technologies: a lack of understanding among technology buyers of what graphics technology can do for them, and the difficulty many graphics platforms have in working seamlessly with third-party libraries and other systems in the world. data processing pipelines.
To address the first issue, the graphing community needs to draw attention to use cases where grafting technologies have been successfully deployed. An example of a use case is cybersecurity. Intruder Detection in computer networks is often performed by creating data origin graphs that track how data is generated and used in computer networks, and then analyzing those graphs for prohibited patterns of data access. Another use case is in the financial services industry, where charting technology can be used to detect financial fraud. The main idea is to build and analyze interaction graphs that connect people, accounts and transactions, then look for patterns such as rings of suspicious transactions to ensure that transactions are legitimate. Highlighting and explaining these kinds of successes is essential to realize the full potential of chart technology in other industries.
But the adoption of graphics technology across industries and sectors is also hampered by the limitations of many existing graphics platforms. Some graphics platforms are single-machine systems, limiting the size of the data sets that can be processed in a reasonable amount of time. The solution is to build scalable graphing platforms in which graphs are shared or distributed across the memories of different machines of a cluster, allowing systems to process much larger data sets more efficiently.
Another problem with many existing graphics platforms is their limited interoperability with other libraries and data processing systems. For example, all major pharmaceutical companies are actively building and operating medical knowledge graphs, but they also make extensive use of third-party chemistry libraries that support functionality such as the calculation of chemical similarity scores. It is essential for the graphics platform to work seamlessly with such libraries because a typical data processing pipeline involves using a graph database query to select a subgraph from the knowledge graph, passing the subgraph to the chemistry library to perform functions for compute the nodes of the graph and then pass the resulting property graph to a graph convolution network that computes a vector space model for the graph.
Another problem is the cooperation with other data processing systems. In many data processing pipelines, graph computing is one of a series of operations that must be performed on the data, while other operations are performed on other big data processing systems. In this context, a lack of seamless integration between systems can become expensive as a lot of time and computation can be spent on transferring data between different systems. An example is an identity management company that attempted to use a graphics technology platform that lacked advanced integration with Spark, a data computing language. The company was still able to get the job done, but with more machines, energy consumption and higher operating costs. After moving to a graphics technology platform with native Spark integration, the customer was able to perform data transfer and analysis more efficiently, significantly reducing costs.
The need for graphics technology is clear – and will grow even greater in the coming years, as 95% of businesses indicate that managing unstructured data is a serious challenge. Highlighting successes and promoting interoperability with other libraries and data processing systems will ensure that graphics technology has a bright future.
About the author
Keshav Pingali is CEO and co-founder of Katana Graph, the AI-powered Graph Intelligence Platform that provides faster, deeper, and more accurate insights into huge and complex data. Keshav holds the WA”Tex” Moncrief Chair of Computing at the University of Texas at Austin, and is a Fellow of the ACM, IEEE, and AAAS.
Sign up for the free insideBIGDATA newsletter.
Follow us on Twitter: @InsideBigData1 – https://twitter.com/InsideBigData1