February 7, 2024
Observability: Choosing The Best Time to Retire Tech

Register to receive insights from CJC

Get notified of the latest news, insights, and upcoming industry events.

Accurate Data Is Mandatory Before Extending Server Lifespan

Steve Moreton

Armed with accurate data, extending server lifespans can provide significant benefits as AWS recently demonstrated, expecting an extra $900m per quarter. However, haphazardly extending lifespans without sufficient data is a potentially costly and dangerous game. Holistic observability is crucial for the acquisition of this data, with the number and size of blind spots corresponding to the level of risk. In this insight, we review:

  1. The Challenge of Market Data Capacity Management.
  2. Is Extending Server Lifespans Safe?
  3. Enhancing Observability, Holistically.

Editor: Antony Fung, Marketing Manager at CJC.


Last year, two of the three largest cloud providers, Google Cloud and Amazon AWS, extended their server lifespans and Waters Technology reached out to discuss the often-overlooked topic1. It is a universal subject all IT engineers are familiar with and was good to see it highlighted. In the case of AWS, the decision makes good business sense as the company subsequently expects to generate an envious extra $900m per quarter. For firms in the capital market, this is a 24x7 subject and focus from a capacity management aspect and is a cornerstone of CJC’s managed services and IT analytics platform – mosaicOA2.

The Challenge of Market Data Capacity Management

Market data consumption and the capital markets create unique capacity management challenges, doubling every two years and is capable of spiking over a millisecond or triple in a day due to market volatility. For example, in the early 2000s, CJC saw Market Data Message Rates between 50-65 billion per day at firms (figure 1). In 2024, this now easily exceeds 170 billion. However, notice at times like COVID-19 and US elections – “Normal” 2024 message rate levels were already seen during volatile 2020 periods. This is a normal occurrence in the capital markets.

Figure 1: Market Data Message Rates seen between 2020-24.

Figure 1: Market Data Message Rates seen between 2020-24.

Aside from market volatility, capital markets are unique in that it has a concept of market opening and closing. Most capital market servers are “asleep” during weekends and evenings when close to no activity takes place. For example, it is easy to distinguish the days, weekends and even holiday periods in Figure 2 (below) but the downtime wreaks havoc on forecast projections.

Figure 2: Market Open vs. Closed Periods.

Figure 2: Market Open vs. Closed Periods.

Is Extending Server Lifespans Safe?

IT managers have often resorted to formulas to work around capacity. For example, investing in servers with the highest specification and networking possible with the latest software deployed, monitoring key application and hardware metrics – refreshing every 3 years. Gambling on extending server lifespan is a potentially dangerous game and accurate data is mandatory before proceeding.

The major cloud providers provide excellent server statistic metrics and Chief Information Officer (CIO) dashboards, however, the dashboards are usually unable to see inside the running market data application. This is an example of an observability blind spot that CJC’s Chief Technology Officer, Peter Williams, touched on3 back in October. Generally, applications crash before a modern CPU hits a high level and knowing these limits or when the system will likely hit them is vital.

Obtaining Observability, Holistically

mosaicOA4 is capable of visualising and illustrating where market data firms are, were, and will be from a capacity management perspective. The data is stored indefinitely in a time-series database, ensuring historical events (like COVID, US elections or announcements) are recorded alongside the system impact.

The busy day view (Figure 3 below) displays both historical and real-time system information, which provides context for the users to know if they are higher/lower than normal. In Figure 3, the system experienced 8 billion more messages than normal and provided hours or even weeks of advanced notice on a potential system crash.

Figure 3: mosaicOA’s Busy Day View.

Figure 3: mosaicOA’s Busy Day View.

Figure 4: mosaicOA’s Busy Day View.

Figure 4: mosaicOA’s Busy Day View.

Additionally, mosaicOA dashboards also indicate which systems are at risk, and which are not (figure 4, right).

The Hot Server Plugin leveraged by mosaicOA uses “principal component analysis5” to provide a server rating over time, which identifies and highlights servers that are over or under-utilised. 

All these techniques combined and visualised in a single interface provide IT managers and executives with the necessary insights to instil system stability confidence with the potential to extend server lifespan – or invest in new technologies. Avoiding unnecessary costs and outages.

CJC's managed services leverage mosaicOA and are procured standalone as-a-service by clients to support RTDS, Exegy, Solace and Kafka. mosaicOA can be used to investigate environments on-premises or in the cloud (Google, Azure, and AWS).

mosaicOA – Global Real-Time IT Monitoring and Observability

mosaicOA is a powerful, cloud-based IT analytics SaaS Solution that consolidates large data sets into a single interface. The solution enables IT teams to easily analyse and gain valuable insights into critical capital markets infrastructure. Features include:

  • Capacity Management.
  • Problem Management.
  • Reporting & Data Visualisation.

About CJC:

CJC is the leading market data technology consultancy and service provider for global financial markets. CJC provides multi-award-winning consultancy, managed services, cloud solutions, observability, and professional commercial management services for mission-critical market data systems. CJC is vendor-neutral and ISO 27001 certified, enabling CJC’s partners the freedom to focus on their core business.

For more information, contact us or:

Email: [email protected]
Tel: +44(0) 203 328 7600

Get In Touch

Get in touch with our experts to learn how we can help you optimize
your market data ecosystem!
Arrange a Meeting