Antony Fung

Download PDF with citations

What's driving organizational efforts to move market data to the cloud? Is a lack of expertise stifling your investment in cloud computing? According to new research from The London School of Economics and Political Science, the answer is probably yes.

The cloud promises improved efficiencies and faster innovation but it doesn't run itself. Recently, we announced that we have completed the migration of all our managed service products and tooling to the public cloud.  Make no mistake, this was a massive project and undertaking but also the same challenge that many capital markets firms are facing and already leveraging CJC for.  CJC is releasing a blog series that covers the following:

• Expertise

• Why and What?

• How?

• Migration and Security

• Benefits and Future

In our previous post in this series, we discussed the expertise CJC has developed in public cloud vendors and technologies to assist firms to transform and migrate their capital markets platforms to the public cloud.  Narratively, you would be forgiven to expect the ‘Why’ as the chronological first step. 

CJC, as engineers first and foremost, know it's best to be experts in new technology before you migrate to it.  Having the expertise helps best inform ‘why’ you move to the cloud but importantly, ‘what’ you can move to the cloud.  Make no mistake, there are many legacy elements that might not work best for the cloud and there are currently latency and networking concerns that must be addressed.

So, why and what did we move?

A review was compiling the following information:

• What we had

•  How it worked

• What were the pain points?

• Would it work in the cloud?

• What could the benefits be?

What we had

CJC had 4 main support centres globally running 24 x 7 (London, Hong Kong, Singapore and New York).  These firms were supporting data centres in almost 30 different geographical locations which could be broken down into 2 categories:

a)  Clients who leverage CJC services, CJC connect to the client’s technologies and IT tooling footprint.

b)  Clients who leverage CJC services, client connects their technologies to our IT tooling footprint.

For category b, clients were leveraging our private cloud platforms which would, in turn, provide monitoring, IT analytics and more on their global infrastructures.  As CJC owned these technologies, this area was placed in scope.  These infrastructures had a significant and prestigious client base grown in just under a decade. 

How it worked

The IT monitoring to client sites worked in real-time with 80 engineers logged securely from all our major centres.  Our clients also securely accessed our footprints from all over the world.  Many firms leveraged our IT analytics product, mosaicOA, of which hundreds of end users access every day for critical dashboards, reporting, capacity management and problem management.  CJC would leverage this for the system we manage, but CJC also has clients who use the product themselves for their own technologies.  This product has a vast array of powerful, time-series databases, all of which have considerable support requirements to ensure seamless delivery of data.

What were the pain points?

The infrastructure was highly available and resilient, however as a private cloud, it was hosted in datacentres such as Equinix.  This came with the requirement for regular smart hands and site visits which needed extensive planning from our service management and PMO teams. 

Our product roadmap included migration to Kafka-based messaging systems and updates to our underlying databases.  This required significant upgrades to our physical technologies, but also came with the added concern that certain software features both known and unknown would not be available. By waiting on physical technology for ‘cloud exclusive’ features, our products would be put on the back foot.

Finally, the public cloud created an issue for the private cloud.  Industry and client demand was clearly showing that the public cloud was the preference.  We now had our global staff supporting two different technology types. Many of our staff were now experienced and accredited in the public cloud. Supporting two types of technologies added demands on our support teams.  It was this element that was a crucial deciding factor.

Would it work in the cloud?

This is where the expertise came in.  Certain elements of our underlying systems were well known to work in the cloud, others were designed for physical technologies.  This hybrid combination of cloud-native/physically written and tuned solutions was designated as a key success criteria.

We needed to ensure that we would be able to match our SLAs and KPIs and not reduce latency on our real-time monitoring to our 80 staff who access the systems.

But the key concern was in terms of handling the big data sets.  Our IT analytics databases all needed to handle hundreds of thousands of updates per second and billions of updates per day – with the data being stored indefinitely.

Finally, if it was practical in the cloud, what would be the migration plan? We had to be certain that there would be a seamless transition from one technology to the other – with no gaps in data.

We also needed to ensure we provide clients the level of security they demand.

To ensure that the above would work and we could achieve a 100% successful migration, a full test, design and customer approved migration plan would be required.  More on that in the next article, however, the answer was a confirmed yes.

What could the benefits be?

Although the benefits of the cloud may seem to be obvious, it's important to detail what we expect them to be for your own infrastructure and products and detail them as success criteria and match them against the completed result.

• Reduced reliance on smart hands-based change work and data centre access

• Increased investment into underlying technologies, adding far greater power, speed and resilience than before

• Ability to include enhancements to underlying technologies which were not available in physical versions

• CJC reduced the amount of infrastructure technology types we support

This detailed review was the foundation of our successful deployment and transition to the cloud.  Migration costs are about ROI – the longer they last, the longer it takes to achieve ROI. 

Cloud assures easy access, speed, efficiency, scalability, better customer experiences and is sustainable in the long run. Also, data and applications in the cloud are better equipped for the constant changes necessitated by the highly evolving technologies. All these points make for a strong case for organizations to move to the cloud.

Our approach has been proven historically since 1999 to be the way to ensure the successful migration of platforms in months rather than years.