Antony Fung

Download PDF with citations

CREATING THE BUILDING BLOCKS FOR MARKET DATA CLOUD EXPERTISE

In 2019, the AFME reported that “Spend on public cloud across all industry sectors is expected to reach $331bn in 2022, compared to $210bn in 2019. Of this, financial services now account for 10.6 percent of this total, as the adoption of public cloud services becomes increasingly important.”

C-level executives are calling for enterprise-wide modernisation and slashing costs, and vendors and regulators are issuing complex auditing requirements. The cloud provides the capital markets the capabilities to address these requirements. Financial institutions are migrating from traditional methods of market data distribution, transforming their legacy data centres architecture to the cloud.

These trends are at the technical heart of CJC. Not only are we actively transforming our clients infrastructures to the cloud but it's also something that we have completed for our technical IT requirements. Recently we announced that we have completed the migration of all managed service products and tooling to the public cloud and adopted a cloud-first strategy. This was a massive project and undertaking, equal to that of our capital markets clients. The expertise, tools and methodologies we used for ourselves, are the same we provide our clients. Therefore, this makes an ideal subject to discuss how we provide transformation and how clients can leverage our capabilities. CJC is releasing a blog series which will cover the following:

 Expertise

• Why?

• How?

• Migration and Security

• Benefits and Future

As mentioned, this was a complicated project:

• We had an established physical data centre footprint in Equinix

• We had an array of managed service clients connected globally

• We had to meet our client’s IT security requirements

• Our clients leveraged complex real-time monitoring systems

• We leveraged our Big Data analytics platform, with many powerful time-series databases

• Migration had to be seamless and without gaps

• We had to have a side-by-side migration and decouple existing infrastructures

However, these challenges are typical when upgrading or migrating platforms. What made this significant was that the underlying technology was migrating to the public cloud.  Public cloud works in a very different way to the physical/virtual-based technologies which are commonplace in the capital markets. This is no ‘lift and shift’ but a transformation of the way the architecture is designed, built and supported. Indeed, challenge is an identified barrier to change when migrating to the cloud. One key thing needs to be in place before migration: expertise.

On a recent webinar, a leading capital markets manager stated: “A Market Data Engineer would make a terrible cloud. A Cloud Engineer would make a terrible market data system”.  This is also backed up by the AFME Report “There may be a shortage of, or limitation on, the availability of subject matter experts (SMEs) within the business or operations who have sufficient understanding to define and realise the target operating model to support public cloud adoption.”

This lack of expertise is an identified challenge to migrating to the cloud. In short, to get cloud, you need to learn it first. 

A key attribute our clients like about us is our ability to spot trends and react to them. As much as we support physical and private infrastructures in 22 geographical locations, CJC has also leveraged the public cloud, such as AWS since 2011.  However, it was in 2017 that we started to ensure that our staff were specifically trained in the public cloud.

Peter Williams, CJC CTO: “In 2017 our first requests to migrate a major capital markets system to public cloud came through the door. We knew that you could not ‘lift and shift’ with cloud at this stage – where you take a physical server and replace it with a physical server, which many firms were adopting.  We also knew that we needed to increase our skillsets in this area – there was no way that these public cloud requests were a flash in the pan.”

In late 2017, CJC started an education process with strategic members of our global Engineering and Operations teams becoming the ‘Cloud Acceleration Team’ (CAT). 

“Back in 2017, The CAT team and myself immersed in public cloud technologies, concepts and innovations and reached out and worked with the major cloud providers globally. We needed to take what we knew, learn new concepts – but also uncover the unknowns to make them skills.  A key output was our strategic view of container-based technologies, such as docker and Kubernetes were going to be a game changer. These concepts would of be great assistance to the capital markets in the transformation of major notable critical messaging platforms and affiliated concepts like databases to a cloud market data platform. This became a big focus.”

It was at this point that CJC began to train staff and focused specifically on Google Cloud Platform to leverage the Kubernetes aspects. To achieve accreditation, CJC needed to achieve a solid percentage of staff trained in GCP and be certified against their platform.  Although the focus was on GCP, CJC continued to use AWS and Azure.

“Clients have a cloud-first policy, so that could be with AWS, Azure and GCP etc. However, our clients have always stated that in the capital markets, they have remit to choose whichever cloud they see fit to be used on the technologies they have – CJC have aligned ourselves to this, our cloud-first policy is with GCP – but we can always leverage the others should they provide the best solution, and we continue to train and learn these infrastructures.”

CJC became Google Partners in Q1 2019 and continues to train and hire staff globally who come with cloud-based skillsets. 

Not only was it important to be accredited, CJC wanted to bring their own cloud-native tooling to the table. CJC built tooling to assist in the complex migration of platforms, products, applications and databases to the cloud.

Although this expertise and tooling is leveraged by firms to assist in their migrations, CJC themselves leveraged physical technologies for their managed services to clients and their big data analytics platform, mosaicOA.

“CJC has positioned and timed itself correctly. We know if we had gone to clients 18 > 12 months ago, they would have not been willing to migrate or transform systems, either ours or theirs to the cloud. The attitude to the cloud has changed significantly in 2020/2021, but vitally, we still needed to demonstrate our mature capabilities and the expertise to achieve this successfully. You need to be able to show firms you have the accreditations and the relationships with these cloud providers.”

In our next article, we will discuss why we moved and how this was done.