The combination of graph database and cloud computing is an inevitable trend. Native and parallel are the core technical advantages, and mo-201 dumps TigerGraph will usher in a comprehensive explosion.
Business growth in the era of big data has brought two obvious changes: the dramatic increase in data volume and the complexity of data associations. At the same time, users' expectations for the value of data are getting higher and higher, and these changes have led to changes and innovations in databases. The graph database provides the most direct expression of related data, and the graph model's natural tolerance for heterogeneous data, making the graph database usher in a state of rapid development.
At the beginning of 2019, the Gartner Data and Analysis Summit listed the graph as one of the top ten data and analysis trends in 2019. It is expected that by 2022, global graph processing and the application of graph data will grow rapidly at a rate of 100% per year. 2020 Annual conservative estimates are $ 8 billion. The market is still in the blue ocean, but competition is spread all over the world.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Image Credit: Gartner's Top Ten Data and Analytics Trends for 2019
In the United States, graph databases are the fastest growing of all databases in recent years. One typical company is TigerGraph. TigerGraph was founded in 2012 and is headquartered in Silicon Valley. On November 8, 2017, TigerGraph released a new generation of enterprise-level real-time graph database platform, which caused a stir in the market. Why is it 5 years between the launch and the first product release? Wu Mingjie, general manager of TigerGraph China, explained, "Based on C ++, it starts from the underlying core, uses distributed technology, supports trillion-level nodes, has high performance and high scalability, and has a very simple operation interface. We use 5 Doom has earned a good reputation for the product. Native and parallel are our differentiation advantages. "
In China, more and more companies have begun to enter the field of graph databases and develop corresponding graph database systems for internal products. However, the application of such graph databases is highly targeted, so they are not suitable for other customers in the market.
Throughout the development of databases, for any data that reaches a certain size or value, a graph database is the best way to present and query these relational data. The ability to understand and analyze these graphs will become the core competitiveness of the company in the future. As Dr. Xu Yu, the founder of TigerGraph, said: "In the era of big data, data is the new oil, and graph-based data analysis is the new refinery."
I. Hot Issues in the First Year of Graph Computing
The focus of graph calculation includes two points: the graph database and the graph analysis engine. The former focuses on OLTP and the latter focuses on offline graph calculation. Of course, the boundary between the two is sometimes not clear. Some graph analysis engines also implement transactions and persistence.
Graph computing has received widespread attention in recent years, mainly due to the following factors:
The data suitable for graph representation has accumulated to a certain extent, and the demand for knowledge mining hidden in graph data has also become stronger and stronger.
Graph neural networks are a major innovation in deep learning. The development of deep learning has encountered a bottleneck, and unexplainability is its inherent ill. The graph calculation combined with deep learning combines end-to-end learning and inductive reasoning, which is expected to solve the inherent problems of relational reasoning and interpretability that deep learning cannot handle.
Knowledge graphs are receiving more and more attention. Certainty knowledge is stored in the graph database in the form of actual vertices and edges. Probabilistic knowledge generally depends on the graph analysis engine for knowledge reasoning. Graph neural networks and knowledge graphs will be opportunities for future detonation graph calculations.
However, to achieve large-scale application of graphs, one of the technical difficulties that must be overcome first is how to improve graph analysis capabilities. Native graph data storage can span data silos, effectively model, explore and query data with complex relationships, and then solve the problem that SQL is not always effective in querying complex data. However, the problem is that the storage and analysis of graph data requires high data regularity. Once the data regularity is poor, it will affect the calculation, and the analysis difficulty will increase as the analysis dimension increases.
2. What is the development of graph computing?
In early 2019, a former Google employee and founder of graph database service company Dgraph, Manish Rai Jain, stood up and reviewed the past and present life of Dgraph, a database product, in the article "10 years, why Google still can't figure out the knowledge graph." By the way, it also revealed that some Google has not been able to get the inside story of the knowledge map for so many years, including internal struggles of the company and the project's development direction has been misled many times. In particular, he pointed out the misunderstanding of most engineers: thinking that a graph is actually a very simple problem that can be solved by building a layer on top of another system.
In fact, the graph calculation engine is the key to the development of graph technology.
From the perspective of the graph technology itself, the entire graph computing field can be further subdivided into sub-directions such as query analysis, calculation engines, storage management, and visualization. There is currently no graph engine that covers all sub-directions. For example, the graph databases Neo4j, Titan, etc. are good at transactional query of graph data, but they cannot perform offline analysis efficiently. The analysis engines Turi and GraphX focus on offline analysis and mining of graph data, but they cannot manage attribute graphs and do not support it. Real-time query. Graph computing engines are diverse. The best known are the in-memory, stand-alone graph computation engines Cassovary and the distributed graph computation engines Pegasus and Giraph.
From the application level, the following problems exist:
1. It is widely used in financial anti-fraud, social, marketing, logistics, and IoT fields, but there are few end-to-end solutions, and the user threshold is high. In addition, the practical accumulation of new technologies is still immature. Take the knowledge graph as an example. In the past few years, everyone discussed the application implementation, so the knowledge graph is on fire. However, when the project is about to land, many technical obstacles are encountered. For example, in terms of performance, data loading is a bottleneck. This results in the knowledge spectrum cannot be updated in real time and many projects cannot be brought online. At present, super-apex problems, OLTP and OLAP fusion problems, graph division, and streaming graph data processing are still the focus of research in academia and industry.
2. From the perspective of enterprise users, the data quality and application methods are immature, and users do not know what they want, which makes it difficult to implement map technology. The implementation process of graph database generally includes 5 steps: business understanding, graph Schema design, business data processing, business verification and monitoring operation and maintenance. The first step is business understanding. This step requires users to clearly understand what they want, understand business data and business problems, map business problems to map problems, and confirm data sources, storage media, update methods, disaster recovery plans, etc. and match them. Corresponding software and hardware resources. Many business users started designing without clearly understanding the business, leading to a dismal end. Therefore, the current application graph database is mainly BATJ, today's headlines, Huawei and other front-line companies, as well as large and medium-sized financial institutions, governments and so on.
3. From the perspective of market competition, there are many different types of graph database products. Each product has different advantages. As graph databases are continuously applied in enterprises and use cases in various vertical industries, it is important to understand the differences. The comparison of the graph database status is shown in the figure below.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Choosing a graph database is mainly considered in terms of performance, query capabilities, and computing capabilities.
On the performance level: the execution of graph query and graph analysis algorithms is fast, and the underlying storage structure is involved. Because native graph storage is based on points and edges, there is no need for too many logical and physical layer transformations in the calculation process. Storage and calculation are parallel and therefore the fastest. The typical representative is TigerGraph.
In terms of query ability, the performance of ordinary graph database querying 3 degrees and above is usually very low, and in fact, it is often necessary to query 6 degree relationships. The test report shows that only TigerGraph completed the 6-degree relational query. In addition, to handle large and growing data sets, TigerGraph is designed to scale horizontally and vertically. Regardless of the number of steps traversed in the graph, the real-time large graph can achieve more than tens of millions of point-edge traversals / second / server, and nearly 10,000 updates / second / server real-time large graph analysis. At the same time, TigerGraph also provides real-time deep correlation analysis, which can realize large graph traversal of more than 10 steps, as well as fast graph traversal and data update.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Graph analysis and calculation capabilities: If the graph database only has storage and query capabilities, it also needs to rely on external GraphX and other calculation engines to complete some graph algorithm analysis, so there is a problem of conversion waste in data transmission and graph expression. TigerGraph can implement stored procedure-like algorithm encapsulation through GSQL, speeding up data loading and algorithm execution speed.
In addition to technical differences, in terms of business concepts, different graph data vendors have different considerations, taking Neo4j and TigerGraph as examples.
Neo4j was born in 2007 and started as an open source project. The development of Neo4j to the present has its inherent advantages, that is, it has caught up with the golden age of open source software, and the entire ecosystem and Neo4j users have contributed a lot of ideas. Visualization, management interface, and good language design are the first advantages of Neo4j. In the early days, because it was open source, everyone was using Cypher, so the basics are better.
On November 15, 2018, Philip Rathle, Vice President of Products for Neo4j, announced that starting with Neo4j 3.5, the Enterprise Edition will only be available under a commercial license, and source code will no longer be available on GitHub. To a certain extent, it is also under the pressure of the rapid growth of the latecomer TigerGraph business.
Dramatically, not long after Neo4j announced the cessation of open source, TigerGraph, the world's fastest enterprise-level graph analysis platform, announced that it has officially open sourced the TigerGraph GSQL graph database algorithm library and released three major ten core algorithms for users to use directly. At the same time, some visualization tools will be introduced one after another to lower the threshold of graph database application.
From the perspective of the business model, TigerGraph did not choose open source at first, mainly from the perspective of enterprise users. For example, financial companies really want to go to a production system. For security reasons, they will definitely choose to pay instead of open source. The reason why TigerGraph later open sourced the graph data algorithm library is also based on the industry's stronger demand for graph databases, hoping to promote the implementation of graph database applications.
If the development of graph calculation is divided into several stages: budding period, development period, maturity period and bubble period, the current graph calculation is still in the development period. In the field of graph databases, it is far from the maturity of the current traditional RDBMS, as shown in the following:
The graph database product is extremely rich, and there are several players with monopolistic status. The performance and functions have completely crushed other products.
Graph data query language completes ISO standardization (OpenCypher and GSQL are currently "absolute winners"), and most products follow this standard, and data exchange standards have emerged.
Users are generally educated, and they can accept graph database applications as naturally as using RDBMS. There are a large number of third-party graph application developers and integrators.
In comparison, the maturity of graph analysis engines is weaker than that of graph databases. At present, there are many academic studies, but there are very few industrial-grade products, and there is no framework and products specifically designed for graph analysis. At present, many front-line companies They still use general computing frameworks such as Spark, Flink, and MapReduce to perform graph calculations. The performance is definitely not as good as a system designed and optimized for graph analysis. Academic systems often focus on a small improvement point and limit their application to specific scenarios. Another biggest problem is that productization is generally weak, and ease of use and fault tolerance are not good. It generally runs on abstract graphs, and users need to use other tools for preprocessing.
3. Graph analysis capabilities become the "competitive sword" of graph database companies
Graph databases have been adopted by companies such as Twitter, Facebook, and Google for a long time, and the technology has been heating up ever since. Cloud service provider giants Amazon, IBM, and Microsoft have added graph databases in the past two years, but when the amount of data becomes very large, or problems require deep correlation analysis, and answers must be provided in real time, most graph databases will Analytical capabilities are at stake.
This is because the architecture of previous generations of graph databases cannot meet the scale and query speed requirements of today's data. First-generation designs (such as Neo4j) were not built around the core of parallelism or distributed database concepts. The second generation features a graph view built on top of NoSQL storage. These products can scale to huge scales, but this additional layer loses huge potential performance. If there is no native graph design, the cost of executing multi-step queries will be high, so many NoSQL platforms can only provide high read performance, and do not support real-time updates.
And these are exactly the advantages of TigerGraph.
TigerGraph breaks through the limitations of previous generations with native parallel graphs, enabling deep association analysis. We know that the native graph database stores data directly in the graph model format (points and edges) and provides built-in indexes that can speed up data loading and build graphs quickly. In contrast, non-native graphs can easily support multi-modal databases, but they sacrifice graph performance. If the data set is large, non-native graphs are usually difficult to handle queries with more than 3 steps.
Scalability is an important feature of the big data era. Enterprises need to scale their database solution to multiple computers because their data may grow too large to be economically stored on a single server. TigerGraph supports scale up and scale out and can automatically divide graph data into one In a server cluster, while maintaining real-time speed, improve load and query throughput. TigerGraph is good at parallel processing while executing each task. It adopts MPP design architecture, and each point and association can be associated with one.
Compute functions, and each point or association can be used as a parallel storage unit at the same time, with compact storage and fast access. TigerGraph uses a distributed computing model that allows all servers to participate in the query. For example, when traversing the path from server A to server B, it will pass to server B the minimum amount of information it needs to know. Server B has already learned After the entire query request, you can easily participate in the division of labor, which significantly improves the performance of analytical queries that traverse most of the graphs.
TigerGraph unifies real-time analytics with large-scale offline processing. Real-time updates mean that database updates can be performed simultaneously with other queries on the database and can be completed quickly. Most non-native graph platforms do not support real-time updates because their data storage systems are immutable. TigerGraph supports real-time analysis of Web-Scale, updating thousands of points and edges every second and hundreds of millions of updates every day. Taking the financial field as an example, the TigerGraph real-time graph database supports storing data from different sources in the graph database for centralized processing, so as to identify fraud before payment processing. Secondly, the TigerGraph graph database implements millisecond-level real-time discrimination based on graph databases by modeling user relationship characteristics. Most importantly, it can provide solutions for the "interpretable AI" that businesses care about.
At the product level, TigerGraph has made strategic adjustments, no longer frantically pursuing extreme speed, but more concerned about the security and robustness of the system. Therefore, TigerGraph has developed a highly available backup component. In addition, TigerGraph spends a lot of time to improve management System (administrator's interface). At present, the administrator's interface is visualized. You can see the machine's occupation of resources and detailed logs. If the system is down, TigerGraph can collect information very clearly, without the need for customers to collect it again. .
The combination of graph database and cloud computing is an inevitable trend. TigerGraph is the first vendor to provide a cloud service for graph databases. The TigerGraph Cloud launched at the end of 2018 can run scalable, simpler and more powerful graph analysis in the cloud. Users can use TigerGraph's customizable graph algorithm The library powers artificial intelligence and machine learning.
TigerGraph is also constantly innovating in graph visualization tools. At present, TigerGraph already has a visually available version. As long as the method is dragged, the system can feedback the results in real time.
In September last year, a project proposal for the GQL (Graphic Query Language) standard was approved, and GQL's graph query language standard is planned to be launched in the second half of this year. GQL was proposed because the original SQL / PGQ was limited to read-only queries. It could not project new graphs. It could only access graphs based on graphical views that generated SQL tables, which limited the development of the graph database market. At present, there are relatively few developers who understand graph query languages on the market, and there is no uniform language standard. GQL will be a milestone in the field of graph databases, marking the further maturity of the graph database market, reducing the learning cost of personnel, and promoting the market More complex talents. Currently TigerGraph is actively promoting the development of GQL. Native parallel graphs and query languages have a natural fusion, which can help existing GSQL users quickly adapt to GQL. This is one of TigerGraph's original intentions to participate in the development of GQL standards.
Summary
Graph computing is in the ascendant. Behind the noise, we should calmly see the difficulties it has not yet broken through, such as the aforementioned super-apex problem, OLTP and OLAP fusion problems, graph division, and streaming graph data processing. At present, knowledge graphs and neural networks on the cusp of the wind are many things worth thinking about.
For example, behind the map becoming a hotspot, we should pay more attention to the application phase of the map. Some stages can not be separated from human participation, and some stages should minimize human intervention, to achieve a certain balance between the two. Now that we are all talking about 5G, is there a connection between 5G and the map? 5G has given birth to new applications of AI. In the future, the imagination space for the map to evolve from single mode to multi-mode is still very large.
In addition, graph neural networks will become the focus of the next few years. Although the graph neural network has not become a phenomenon-level technology in the industry, it has been tried in large companies. The first point mentioned by Alibaba in the 2020 Top 10 Trends in Dharma Institute 2020 is the evolution of artificial intelligence from cognitive intelligence to cognitive intelligence. Large-scale graph neural networks are considered to be powerful in promoting cognitive intelligence. Method of reasoning. However, we must see that the implementation of graph neural networks is still in its early stages. Although the software and hardware facilities accumulated in the previous wave of deep learning have laid the foundation for the implementation of graph neural networks, there is still a gap between it and the ideal.
In 2019, hard technology encounters various practical barriers. mo-201 dumps Although there are limitations and helplessness, it still cannot stop the development of greater scientific and technological forces. As a point put forward in Singularity Approaching: "Our future is not going to undergo evolution anymore, it is going to undergo an explosion."
2020 will also be a year of macro exhibitions.
Business growth in the era of big data has brought two obvious changes: the dramatic increase in data volume and the complexity of data associations. At the same time, users' expectations for the value of data are getting higher and higher, and these changes have led to changes and innovations in databases. The graph database provides the most direct expression of related data, and the graph model's natural tolerance for heterogeneous data, making the graph database usher in a state of rapid development.
At the beginning of 2019, the Gartner Data and Analysis Summit listed the graph as one of the top ten data and analysis trends in 2019. It is expected that by 2022, global graph processing and the application of graph data will grow rapidly at a rate of 100% per year. 2020 Annual conservative estimates are $ 8 billion. The market is still in the blue ocean, but competition is spread all over the world.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Image Credit: Gartner's Top Ten Data and Analytics Trends for 2019
In the United States, graph databases are the fastest growing of all databases in recent years. One typical company is TigerGraph. TigerGraph was founded in 2012 and is headquartered in Silicon Valley. On November 8, 2017, TigerGraph released a new generation of enterprise-level real-time graph database platform, which caused a stir in the market. Why is it 5 years between the launch and the first product release? Wu Mingjie, general manager of TigerGraph China, explained, "Based on C ++, it starts from the underlying core, uses distributed technology, supports trillion-level nodes, has high performance and high scalability, and has a very simple operation interface. We use 5 Doom has earned a good reputation for the product. Native and parallel are our differentiation advantages. "
In China, more and more companies have begun to enter the field of graph databases and develop corresponding graph database systems for internal products. However, the application of such graph databases is highly targeted, so they are not suitable for other customers in the market.
Throughout the development of databases, for any data that reaches a certain size or value, a graph database is the best way to present and query these relational data. The ability to understand and analyze these graphs will become the core competitiveness of the company in the future. As Dr. Xu Yu, the founder of TigerGraph, said: "In the era of big data, data is the new oil, and graph-based data analysis is the new refinery."
I. Hot Issues in the First Year of Graph Computing
The focus of graph calculation includes two points: the graph database and the graph analysis engine. The former focuses on OLTP and the latter focuses on offline graph calculation. Of course, the boundary between the two is sometimes not clear. Some graph analysis engines also implement transactions and persistence.
Graph computing has received widespread attention in recent years, mainly due to the following factors:
The data suitable for graph representation has accumulated to a certain extent, and the demand for knowledge mining hidden in graph data has also become stronger and stronger.
Graph neural networks are a major innovation in deep learning. The development of deep learning has encountered a bottleneck, and unexplainability is its inherent ill. The graph calculation combined with deep learning combines end-to-end learning and inductive reasoning, which is expected to solve the inherent problems of relational reasoning and interpretability that deep learning cannot handle.
Knowledge graphs are receiving more and more attention. Certainty knowledge is stored in the graph database in the form of actual vertices and edges. Probabilistic knowledge generally depends on the graph analysis engine for knowledge reasoning. Graph neural networks and knowledge graphs will be opportunities for future detonation graph calculations.
However, to achieve large-scale application of graphs, one of the technical difficulties that must be overcome first is how to improve graph analysis capabilities. Native graph data storage can span data silos, effectively model, explore and query data with complex relationships, and then solve the problem that SQL is not always effective in querying complex data. However, the problem is that the storage and analysis of graph data requires high data regularity. Once the data regularity is poor, it will affect the calculation, and the analysis difficulty will increase as the analysis dimension increases.
2. What is the development of graph computing?
In early 2019, a former Google employee and founder of graph database service company Dgraph, Manish Rai Jain, stood up and reviewed the past and present life of Dgraph, a database product, in the article "10 years, why Google still can't figure out the knowledge graph." By the way, it also revealed that some Google has not been able to get the inside story of the knowledge map for so many years, including internal struggles of the company and the project's development direction has been misled many times. In particular, he pointed out the misunderstanding of most engineers: thinking that a graph is actually a very simple problem that can be solved by building a layer on top of another system.
In fact, the graph calculation engine is the key to the development of graph technology.
From the perspective of the graph technology itself, the entire graph computing field can be further subdivided into sub-directions such as query analysis, calculation engines, storage management, and visualization. There is currently no graph engine that covers all sub-directions. For example, the graph databases Neo4j, Titan, etc. are good at transactional query of graph data, but they cannot perform offline analysis efficiently. The analysis engines Turi and GraphX focus on offline analysis and mining of graph data, but they cannot manage attribute graphs and do not support it. Real-time query. Graph computing engines are diverse. The best known are the in-memory, stand-alone graph computation engines Cassovary and the distributed graph computation engines Pegasus and Giraph.
From the application level, the following problems exist:
1. It is widely used in financial anti-fraud, social, marketing, logistics, and IoT fields, but there are few end-to-end solutions, and the user threshold is high. In addition, the practical accumulation of new technologies is still immature. Take the knowledge graph as an example. In the past few years, everyone discussed the application implementation, so the knowledge graph is on fire. However, when the project is about to land, many technical obstacles are encountered. For example, in terms of performance, data loading is a bottleneck. This results in the knowledge spectrum cannot be updated in real time and many projects cannot be brought online. At present, super-apex problems, OLTP and OLAP fusion problems, graph division, and streaming graph data processing are still the focus of research in academia and industry.
2. From the perspective of enterprise users, the data quality and application methods are immature, and users do not know what they want, which makes it difficult to implement map technology. The implementation process of graph database generally includes 5 steps: business understanding, graph Schema design, business data processing, business verification and monitoring operation and maintenance. The first step is business understanding. This step requires users to clearly understand what they want, understand business data and business problems, map business problems to map problems, and confirm data sources, storage media, update methods, disaster recovery plans, etc. and match them. Corresponding software and hardware resources. Many business users started designing without clearly understanding the business, leading to a dismal end. Therefore, the current application graph database is mainly BATJ, today's headlines, Huawei and other front-line companies, as well as large and medium-sized financial institutions, governments and so on.
3. From the perspective of market competition, there are many different types of graph database products. Each product has different advantages. As graph databases are continuously applied in enterprises and use cases in various vertical industries, it is important to understand the differences. The comparison of the graph database status is shown in the figure below.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Choosing a graph database is mainly considered in terms of performance, query capabilities, and computing capabilities.
On the performance level: the execution of graph query and graph analysis algorithms is fast, and the underlying storage structure is involved. Because native graph storage is based on points and edges, there is no need for too many logical and physical layer transformations in the calculation process. Storage and calculation are parallel and therefore the fastest. The typical representative is TigerGraph.
In terms of query ability, the performance of ordinary graph database querying 3 degrees and above is usually very low, and in fact, it is often necessary to query 6 degree relationships. The test report shows that only TigerGraph completed the 6-degree relational query. In addition, to handle large and growing data sets, TigerGraph is designed to scale horizontally and vertically. Regardless of the number of steps traversed in the graph, the real-time large graph can achieve more than tens of millions of point-edge traversals / second / server, and nearly 10,000 updates / second / server real-time large graph analysis. At the same time, TigerGraph also provides real-time deep correlation analysis, which can realize large graph traversal of more than 10 steps, as well as fast graph traversal and data update.
å ›¾æ • ° æ ®åº" ç «žäº ‰ æ— ¥ 趋 æ¿ € çƒˆï¼Œæ · ˜æ ± ° æ £ åœ¨è¿ ›è¡Œ
Graph analysis and calculation capabilities: If the graph database only has storage and query capabilities, it also needs to rely on external GraphX and other calculation engines to complete some graph algorithm analysis, so there is a problem of conversion waste in data transmission and graph expression. TigerGraph can implement stored procedure-like algorithm encapsulation through GSQL, speeding up data loading and algorithm execution speed.
In addition to technical differences, in terms of business concepts, different graph data vendors have different considerations, taking Neo4j and TigerGraph as examples.
Neo4j was born in 2007 and started as an open source project. The development of Neo4j to the present has its inherent advantages, that is, it has caught up with the golden age of open source software, and the entire ecosystem and Neo4j users have contributed a lot of ideas. Visualization, management interface, and good language design are the first advantages of Neo4j. In the early days, because it was open source, everyone was using Cypher, so the basics are better.
On November 15, 2018, Philip Rathle, Vice President of Products for Neo4j, announced that starting with Neo4j 3.5, the Enterprise Edition will only be available under a commercial license, and source code will no longer be available on GitHub. To a certain extent, it is also under the pressure of the rapid growth of the latecomer TigerGraph business.
Dramatically, not long after Neo4j announced the cessation of open source, TigerGraph, the world's fastest enterprise-level graph analysis platform, announced that it has officially open sourced the TigerGraph GSQL graph database algorithm library and released three major ten core algorithms for users to use directly. At the same time, some visualization tools will be introduced one after another to lower the threshold of graph database application.
From the perspective of the business model, TigerGraph did not choose open source at first, mainly from the perspective of enterprise users. For example, financial companies really want to go to a production system. For security reasons, they will definitely choose to pay instead of open source. The reason why TigerGraph later open sourced the graph data algorithm library is also based on the industry's stronger demand for graph databases, hoping to promote the implementation of graph database applications.
If the development of graph calculation is divided into several stages: budding period, development period, maturity period and bubble period, the current graph calculation is still in the development period. In the field of graph databases, it is far from the maturity of the current traditional RDBMS, as shown in the following:
The graph database product is extremely rich, and there are several players with monopolistic status. The performance and functions have completely crushed other products.
Graph data query language completes ISO standardization (OpenCypher and GSQL are currently "absolute winners"), and most products follow this standard, and data exchange standards have emerged.
Users are generally educated, and they can accept graph database applications as naturally as using RDBMS. There are a large number of third-party graph application developers and integrators.
In comparison, the maturity of graph analysis engines is weaker than that of graph databases. At present, there are many academic studies, but there are very few industrial-grade products, and there is no framework and products specifically designed for graph analysis. At present, many front-line companies They still use general computing frameworks such as Spark, Flink, and MapReduce to perform graph calculations. The performance is definitely not as good as a system designed and optimized for graph analysis. Academic systems often focus on a small improvement point and limit their application to specific scenarios. Another biggest problem is that productization is generally weak, and ease of use and fault tolerance are not good. It generally runs on abstract graphs, and users need to use other tools for preprocessing.
3. Graph analysis capabilities become the "competitive sword" of graph database companies
Graph databases have been adopted by companies such as Twitter, Facebook, and Google for a long time, and the technology has been heating up ever since. Cloud service provider giants Amazon, IBM, and Microsoft have added graph databases in the past two years, but when the amount of data becomes very large, or problems require deep correlation analysis, and answers must be provided in real time, most graph databases will Analytical capabilities are at stake.
This is because the architecture of previous generations of graph databases cannot meet the scale and query speed requirements of today's data. First-generation designs (such as Neo4j) were not built around the core of parallelism or distributed database concepts. The second generation features a graph view built on top of NoSQL storage. These products can scale to huge scales, but this additional layer loses huge potential performance. If there is no native graph design, the cost of executing multi-step queries will be high, so many NoSQL platforms can only provide high read performance, and do not support real-time updates.
And these are exactly the advantages of TigerGraph.
TigerGraph breaks through the limitations of previous generations with native parallel graphs, enabling deep association analysis. We know that the native graph database stores data directly in the graph model format (points and edges) and provides built-in indexes that can speed up data loading and build graphs quickly. In contrast, non-native graphs can easily support multi-modal databases, but they sacrifice graph performance. If the data set is large, non-native graphs are usually difficult to handle queries with more than 3 steps.
Scalability is an important feature of the big data era. Enterprises need to scale their database solution to multiple computers because their data may grow too large to be economically stored on a single server. TigerGraph supports scale up and scale out and can automatically divide graph data into one In a server cluster, while maintaining real-time speed, improve load and query throughput. TigerGraph is good at parallel processing while executing each task. It adopts MPP design architecture, and each point and association can be associated with one.
Compute functions, and each point or association can be used as a parallel storage unit at the same time, with compact storage and fast access. TigerGraph uses a distributed computing model that allows all servers to participate in the query. For example, when traversing the path from server A to server B, it will pass to server B the minimum amount of information it needs to know. Server B has already learned After the entire query request, you can easily participate in the division of labor, which significantly improves the performance of analytical queries that traverse most of the graphs.
TigerGraph unifies real-time analytics with large-scale offline processing. Real-time updates mean that database updates can be performed simultaneously with other queries on the database and can be completed quickly. Most non-native graph platforms do not support real-time updates because their data storage systems are immutable. TigerGraph supports real-time analysis of Web-Scale, updating thousands of points and edges every second and hundreds of millions of updates every day. Taking the financial field as an example, the TigerGraph real-time graph database supports storing data from different sources in the graph database for centralized processing, so as to identify fraud before payment processing. Secondly, the TigerGraph graph database implements millisecond-level real-time discrimination based on graph databases by modeling user relationship characteristics. Most importantly, it can provide solutions for the "interpretable AI" that businesses care about.
At the product level, TigerGraph has made strategic adjustments, no longer frantically pursuing extreme speed, but more concerned about the security and robustness of the system. Therefore, TigerGraph has developed a highly available backup component. In addition, TigerGraph spends a lot of time to improve management System (administrator's interface). At present, the administrator's interface is visualized. You can see the machine's occupation of resources and detailed logs. If the system is down, TigerGraph can collect information very clearly, without the need for customers to collect it again. .
The combination of graph database and cloud computing is an inevitable trend. TigerGraph is the first vendor to provide a cloud service for graph databases. The TigerGraph Cloud launched at the end of 2018 can run scalable, simpler and more powerful graph analysis in the cloud. Users can use TigerGraph's customizable graph algorithm The library powers artificial intelligence and machine learning.
TigerGraph is also constantly innovating in graph visualization tools. At present, TigerGraph already has a visually available version. As long as the method is dragged, the system can feedback the results in real time.
In September last year, a project proposal for the GQL (Graphic Query Language) standard was approved, and GQL's graph query language standard is planned to be launched in the second half of this year. GQL was proposed because the original SQL / PGQ was limited to read-only queries. It could not project new graphs. It could only access graphs based on graphical views that generated SQL tables, which limited the development of the graph database market. At present, there are relatively few developers who understand graph query languages on the market, and there is no uniform language standard. GQL will be a milestone in the field of graph databases, marking the further maturity of the graph database market, reducing the learning cost of personnel, and promoting the market More complex talents. Currently TigerGraph is actively promoting the development of GQL. Native parallel graphs and query languages have a natural fusion, which can help existing GSQL users quickly adapt to GQL. This is one of TigerGraph's original intentions to participate in the development of GQL standards.
Summary
Graph computing is in the ascendant. Behind the noise, we should calmly see the difficulties it has not yet broken through, such as the aforementioned super-apex problem, OLTP and OLAP fusion problems, graph division, and streaming graph data processing. At present, knowledge graphs and neural networks on the cusp of the wind are many things worth thinking about.
For example, behind the map becoming a hotspot, we should pay more attention to the application phase of the map. Some stages can not be separated from human participation, and some stages should minimize human intervention, to achieve a certain balance between the two. Now that we are all talking about 5G, is there a connection between 5G and the map? 5G has given birth to new applications of AI. In the future, the imagination space for the map to evolve from single mode to multi-mode is still very large.
In addition, graph neural networks will become the focus of the next few years. Although the graph neural network has not become a phenomenon-level technology in the industry, it has been tried in large companies. The first point mentioned by Alibaba in the 2020 Top 10 Trends in Dharma Institute 2020 is the evolution of artificial intelligence from cognitive intelligence to cognitive intelligence. Large-scale graph neural networks are considered to be powerful in promoting cognitive intelligence. Method of reasoning. However, we must see that the implementation of graph neural networks is still in its early stages. Although the software and hardware facilities accumulated in the previous wave of deep learning have laid the foundation for the implementation of graph neural networks, there is still a gap between it and the ideal.
In 2019, hard technology encounters various practical barriers. mo-201 dumps Although there are limitations and helplessness, it still cannot stop the development of greater scientific and technological forces. As a point put forward in Singularity Approaching: "Our future is not going to undergo evolution anymore, it is going to undergo an explosion."
2020 will also be a year of macro exhibitions.
Comments
Post a Comment