Dolphinscheduler DAG Core Source Code Analysis

پس زمینه

نکته: در Dolphinscheduler، کارهای آفلاین دارای چرخه حیات کامل هستند، مانند توقف، مکث، از سرگیری توقف، اجرا مجدد و غیره، همگی در قالب DAG (Directed Acyclic Graph) برای کارهای آفلاین T+1 سازماندهی می شوند.

اجرای Dolphinscheduler DAG

org.apache.dolphinscheduler.common.graph.DAG

سه ساختار داده مهم DAG:

// Vertex information
private final Map nodesMap;

// Edge association information, which records the relationship between vertices and edges, allowing to find leaf nodes and downstream nodes
private final Map edgesMap;

// Reverse edge association information, which allows for quick finding of nodes with an in-degree of 0 (starting nodes), and also to obtain upstream nodes
private final Map reverseEdgesMap;

مثال زیر:

DAG graph = new DAG();
graph.addNode("A", "A");
graph.addNode("B", "B");
graph.addNode("C", "C");

// Add an edge from B to C, A is still floating
graph.addEdge("B", "C");

// If you add A - B, it actually starts from B and checks if there is a connectable line to A. If there is, it means the A - B edge cannot be added because it would form a cycle; otherwise, it can be added.
graph.addEdge("A", "B");

تجزیه و تحلیل کد منبع: org.apache.dolphinscheduler.common.graph.DAG#addEdge

public boolean addEdge(Node fromNode,...