Grounded Approaches to an Abstract World: Data Analytics, Automation, and Visualization Solutions for Managing Virtual Networks


“The abstract model allows a programmer to build into the software an end-to-end service that would previously have been delivered by deploying physical infrastructure.”

A large assortment of new methodologies – ‘softwarization,’ virtualization, self-organization, automation, APIs, clouds, and more – have promised to streamline network operations. The savings seem to be real and the impact on network operations, the associated tools, and processes is tremendous. In a perfect world, these new technologies would allow the network to take care of itself—adapt, self-organize, self-optimize, and self-heal. But we don't live in a perfect world.

Operators need to evaluate several different tools and approaches, including data analytics, automation, and visualization, to help them reap the benefits of these new methodologies. At the same time, though, a certain level of oversight for when issues do occur must be maintained.

For example, software defined networking (SDN) makes use of abstraction to provide cost savings and new services. SDN provides a way for controllers to manage traffic flows through a network of switches. This is achieved by delivering an abstract model of the network topology to network applications running on the central server.

The abstract model allows a programmer to build an end-to-end service into the software that would previously have been delivered by deploying physical infrastructure. Now, instead of a physical design with cables, routers, and servers, the programmer builds a virtual network out of abstracted virtual connections, virtual routers, and virtual servers. This enables the fast-paced creation and update of virtual network models and services over a physical network that changes at a much slower pace.

So, while abstractions are helpful at improving network operations, they also make it hard to understand what physical resources are really used. For example:

  • Which link on the network introduces latency?

  • How will you know which flows and end-users might be affected by maintenance?

  • If a user requests additional capacity, how would you know if the network has enough spare capacity to handle the additional as well as the legacy traffic?

To address these questions, as well as the larger, underlying issues surrounding abstraction in general, there are a variety of available tools and approaches. Three of the primary methods are data analytics, automation, and visualization.


The first option that could help operators manage these new network technologies could be to implement a data analytics solution. While the volume of data in the network and its traffic is enormous, big data applications are now emerging to help process and extract meaning from diverse sets of information. The focus must not just be about dealing with a high volume of data, but also about processing that data through Artificial Intelligence for meaningful insights on network activities.

But Artificial Intelligence requires entrusting a machine to make decisions. While machines are extremely efficient at resolving problems that are well-defined, they cannot effectively deal with situations that were not foreseen by the programmer or not observed in the data.


Another emerging approach centers around automation. This begins by building models of the relationship between the virtual/logical connections and the physical resources supporting them. Because the connection is complex and changing, and the routing tables and policies may be dynamic, the paths that flow through the network may change. Tools are needed to audit the flows on the network, the status of the different databases, and the utilization of resources.

Some of these tasks can be automated by external audit programs to make sure that databases are consistent, resource utilization is consistent with demand, and performance is at the appropriate level. The most advanced solutions can also offer automated resolution to common problems through closed-loop orchestration, allowing a system to scale up or scale down as needed.


One of the newest approaches is the visualization of the physical and virtual resources and components of the network. This requires providing the technician with a diagram of the virtual network that is being built, the applications, the controllers, the vSwitches, and the vRouter. From that diagram, the technician can select a function and view the physical resources that are supporting it, or vice versa.


In more complex environments, the benefits of visualization are even greater. For instance, when a service spans legacy Physical Network Functions (PNF) and new Virtual Network Functions (VNF), it would be of tremendous help to be able to see what part of the chain is virtual and what part is physical. Similarly, when services in a hybrid cloud cross the boundary between private and public cloud, it would be helpful to visualize which functions are executed where and eventually to visualize the impact of different scenarios.

Such functionality will enable a technician to quickly troubleshoot a problem in the network. As an example, consider how a technician could approach the problem of performance degradation on a customer’s service. The technician could view the abstract network, select the resource that is experiencing difficulty (i.e. view the associated logs and alarms), and drill down to view the physical resources that support it and their associated status.

Visualization will also help the technician perform a root cause analysis. It could be used to display information about the context of a problem, its onset, its duration, its extent, and correlate that to recent events. In this way, a user can quickly narrow down the source of a performance degradation to network aspects. Human oversight through visualization can be compatible with automation, which enhances it by ensuring that even unforeseen events can be dealt with effectively.

While abstractions, such as SDN, will help manage network complexity and improve performance, it is important to maintain some oversight. There are a variety of ways to approach this issue.

Data analytics can provide one avenue to better understanding today’s networks, although such a method may run into issues when unknown scenarios are encountered. Visualization, combined with automation, could offer the best all-around solution by mapping a high-level abstraction into better-defined physical implementations. But, no matter what solution you use, it is vital to understand that we don't live in a perfect world and your network performance will benefit from a grounded approach to abstract environments.