In short, no visual analytics system is truly complete without a heatmap. At CJC, our IT analytics tool, mosaicOA, naturally has this in its wheelhouse – but with its capital markets history and client base, heatmaps are something, we here at CJC, take very seriously. Like all mosaic front end features, heatmaps were designed and developed with Misha Kipnis, CEO of Corellasoft and of course, the historical founder of ITRS Group.

Threshold Challenges for Capacity Heatmap

CJC provides managed services for many capital markets firms; these clients could have 20 servers or up to 4000 servers. No matter where our clients sit, there are thousands of statistics generated from hardware, networking, OS, and application components. Frequently, these components adhere to their own measurement spectrums; for example:

·       CPU:  0 – 100% (with 100% being bad)

·       Memory Availability:  100 - 0% (with 100% being good)

·       Network Card: 0 – 204,799k (based on 25GB network card)

These differences impact thresholds; a threshold is important for defining when a green begins to turn into a red. We can’t repurpose a threshold for CPU which also applies to networking – anything over 100 will be red. Application statistics compound these differences, especially from microservices-enabled capital markets technology.

·       TREP Market Data Update Rates* – 0 – 2.5 million per second

·       Solace Client Messages – 0 - 95 billion

·       Microsecond Latency 0 – X million

*average ADS usage in many capital markets firms. Benchmarked up to 75 million.

Adding to the complexity, as you move around a client infrastructure from a business unit or global perspective, varying levels of usage and scale become factors. This could be a Fixed Income team using their infrastructure more than equities or a US bank having a smaller infrastructure in EMEA or APAC than they do in the AMERS region.

Another element to be considered is ‘license’ thresholds. This could be an agreed/purchased number of users or total market data instruments - these can vary around a site. We must also consider exclusion periods such as ‘out of market hours’ in the evenings and weekends; numbers generated during these times could be due to testing or power-downs.

Threshold Analysis

mosaicOA provides accurate and long-term visualizations to view of all system behaviour from baseline CPU to advanced application-level statistics. The InfluxDB time-series database will chart the entire lifespan of a server’s baseline or application behaviour:

By viewing this data in this manner, you can define what thresholds servers and applications require with confidence.

Building the Heatmap

All IT metrics created by infrastructure are stored to the powerful, time-series database, InfluxDB. The data is stored indefinitely, albeit with some granularity reductions added over a 6-month period. The spectrum of time set to the heatmap is an important factor as this could be the last second to the last year, providing different heatmap results.

The heatmap function is requested via a drop-down in the front end. Users can simply place into the heatmap a query of what servers, components, or application elements need to be visualized. To keep it simple – let’s focus on some baseline server elements:

·       CPU Percent Utilisation

·       Memory Available Percent

·       Network Packet Send

·       Network Packet Receive

Using mosaicOA, we can very quickly request the system and elements into the heatmap view:

Raw Heatmap – no warning