3D Network Visualisations in Deepnote

Jul 30, 2020 by Oliver Gardiner for Deepnote

In this article I am going to take you through how to create interactive 3D network visualisations in Deepnote using Plotly. This is an image of the beautiful final 3D visualisation of Zachary's Karate Club that we are working towards:

This article is going to be all about network visualisations, or more specifically how to create ineractive 3D networks in deepnote. We are going to do this using the python packages networkx and Plotly. If you are interestd in the fundamentals of networks and their potential applications do have a look at my last article on social network analysis.

We are going to walk through how to produce a 3D visualisation of the Zachary Karate Club graph that we worked with in my last article.

In my last post we used networkx to make our visualisations. That just about did the job, but now are going to explore using Plotly to make some more sophisticated visualisations. The open source software Gephi is also often used for visualisations, but Plotly is usually easier to use as it means we can do everything in python and never have to leave our Deepnote notebook!

Set-Up

The first step is to get all of the python packages we need installed and imported. We will also open the Zachary's Karate Club (ZKC) graph, and set-up some variables to hold other useful information.

#Install needed packages !pip install networkx !pip install plotly==4.9.0

#Import the required packages import networkx as nx import plotly.graph_objects as go import pandas as pd %matplotlib inline

#Let's import the ZKC graph: ZKC_graph = nx.karate_club_graph() #Let's keep track of which nodes represent John A and Mr Hi Mr_Hi = 0 John_A = 33 #remember the number of nodes since this will come in useful later Num_nodes = 34 #get the club labels - i.e. which club each individual ended up joining club_labels = list(nx.get_node_attributes(ZKC_graph,'club').values())

In the visualisation we want to distinguish between the different communities detected in my last article. We want to visualise these two communities (our predictions for which faction each individual will join) in a different colour. For ease they are included here:

#communities from last time - one list for each community community_0 = [8, 14, 15, 18, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33] community_1 = [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 16, 17, 19, 21] #label for each node corresponds to community 0 or community 1 community_label = [1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Visualisation using NetworkX

As a reminder of what we are able to do using networkx lets create a quick 2D visualisation.

A convenient feature of using networkx for plots is that we can very easily choose the positions of our nodes using different networkx functions. Last time we used a circular layout, this time we will use a spring layout.

#Let's opt for a spring layout spring_pos = nx.spring_layout(ZKC_graph, seed=2)

#draw the network nx.draw_networkx_nodes(ZKC_graph, spring_pos, nodelist=community_0, node_color='g', alpha=0.4) nx.draw_networkx_nodes(ZKC_graph, spring_pos, nodelist=community_1, node_color='m', alpha=0.4) #let's highlight Mr Hi (solid purple) and John A (solid green) nx.draw_networkx_nodes(ZKC_graph, spring_pos, nodelist=[John_A], node_color='g', alpha=1) nx.draw_networkx_nodes(ZKC_graph, spring_pos, nodelist=[Mr_Hi], node_color='m', alpha=1) nx.draw_networkx_edges(ZKC_graph, spring_pos,stlye='dashed',width = 0.5)

The benefit of this spring layout is that it makes the distinctions between the two detected communities very clear. We can see that both of the communities are clustered around their 'leader' (Mr Hi or John A.).

This matplotlib plot however is ultimately limited, it is tightly packed and it's hard to tell exactly what is going on. To improve on this let's see what can be done with Plotly.

On to Plotly!

It's really easy to use Plotly in deepnote to create a 3D visualisation of a network. Essentially, we are going to make seperate 3D scatter plots (or traces in common Plotly terminology) of the nodes and the edges which will then be plotted together.

#As before we use networkx to determine node positions. We want to do the same spring layout but in 3D spring_3D = nx.spring_layout(ZKC_graph,dim=3, seed=18) #an example node coordinate spring_3D[4]

#we need to seperate the X,Y,Z coordinates for Plotly x_nodes = [spring_3D[i][0] for i in range(Num_nodes)]# x-coordinates of nodes y_nodes = [spring_3D[i][1] for i in range(Num_nodes)]# y-coordinates z_nodes = [spring_3D[i][2] for i in range(Num_nodes)]# z-coordinates

#We also need a list of edges to include in the plot edge_list = ZKC_graph.edges() edge_list

#we need to create lists that contain the starting and ending coordinates of each edge. x_edges=[] y_edges=[] z_edges=[] #need to fill these with all of the coordiates for edge in edge_list: #format: [beginning,ending,None] x_coords = [spring_3D[edge[0]][0],spring_3D[edge[1]][0],None] x_edges += x_coords y_coords = [spring_3D[edge[0]][1],spring_3D[edge[1]][1],None] y_edges += y_coords z_coords = [spring_3D[edge[0]][2],spring_3D[edge[1]][2],None] z_edges += z_coords

#create a trace for the edges trace_edges = go.Scatter3d(x=x_edges, y=y_edges, z=z_edges, mode='lines', line=dict(color='black', width=2), hoverinfo='none')

Before diving into creating a trace for the nodes I want to highlight one of Plotly's most useful features: Hover Labels. It lets the user to reveal more information about a data point by moving their mouse cursor over the point and having a hover label appear. Here we set the hoverinfo to 'text' and the text to the club labels that we retreived at the beginning.

That means in the visualisation we will be able to find out which faction an individual ended up joining just by hovering our mouse close to the node.

#create a trace for the nodes trace_nodes = go.Scatter3d(x=x_nodes, y=y_nodes, z=z_nodes, mode='markers', marker=dict(symbol='circle', size=10, color=community_label, #color the nodes according to their community colorscale=['lightgreen','magenta'], #either green or mageneta line=dict(color='black', width=0.5)), text=club_labels, hoverinfo='text')

To clearly highlight Mr Hi and John A. We need to plot their nodes again in another colour. The simplest way to do this is to create another trace for each node. These will then be plotted on top of the other nodes.

trace_MrHi = go.Scatter3d(x=[x_nodes[Mr_Hi]], y=[y_nodes[Mr_Hi]], z=[z_nodes[Mr_Hi]], mode='markers', name='Mr_Hi', marker=dict(symbol='circle', size=10, color='darkmagenta', line=dict(color='black', width=0.5) ), text = ['Mr_Hi'], hoverinfo = 'text')

trace_JohnA = go.Scatter3d(x=[x_nodes[John_A]], y=[y_nodes[John_A]], z=[z_nodes[John_A]], mode='markers', name='John_A', marker=dict(symbol='circle', size=10, color='green', line=dict(color='black', width=0.5) ), text = ['Officer'], hoverinfo = 'text')

#we need to set the axis for the plot axis = dict(showbackground=False, showline=False, zeroline=False, showgrid=False, showticklabels=False, title='')

#also need to create the layout for our plot layout = go.Layout(title="Two Predicted Factions of Zachary's Karate Club", width=650, height=625, showlegend=False, scene=dict(xaxis=dict(axis), yaxis=dict(axis), zaxis=dict(axis), ), margin=dict(t=100), hovermode='closest')

Now we have got everything sorted here is our plot!

#Include the traces we want to plot and create a figure data = [trace_edges, trace_nodes, trace_MrHi, trace_JohnA] fig = go.Figure(data=data, layout=layout) fig.show()

Great! So there we have it. A 3D visualisation of our Zachary Karate Club graph. The two communities (purple and green) represent our predictions for which faction each individual will join. Whilst the dark purple and dark green nodes represent Mr Hi and John A respectively.

If you hover your mouse over the different nodes it will also tell you which faction the individual actually chose to join. In my last article our predictions were wrong for two individuals. See if you can find them (hint they are near the centre of the graph).

What about a more complicated example?

The internet is a typical example when it comes to complex networks. In fact, it is this insight, modelling the internet as a network, that allows Google to return effective search results.

Below is a 3D visualisation of a network 2000 different webpages, generated from networkx (feel free to play around with it, it's also interactive!). The colors represent different communities that were detected.

That's it for this article, but I hope it has given you everything you need to know to open up your own Deepnote notebook and have a play around with your own visualisations!

3D Network Visualisations in Deepnote

Set-Up

Visualisation using NetworkX

On to Plotly!

Now we have got everything sorted here is our plot!

What about a more complicated example?

Recommended on Deepnote

Stock Market Analysis

The 10 Best Ways to Create NumPy Arrays

Wide Residual Networks