Transitioning to the cloud is a non-trivial decision for most organisations, and those responsible and accountable for making such a decision must evaluate the data and service that they plan to migrate to the cloud.

Questions to consider include:

• How sensitive is the data, and what are the necessary minimum security controls? 

• Do privacy restrictions apply?

• Where is the data encrypted?

• Is the cloud contract fit for purpose and compliant with all application regulations?

Recently we announced that we have completed the migration of all managed service products and tooling to the public cloud.  Make no mistake, this was a massive project and undertaking – but also the exact same challenge that many capital markets firms are facing – and already leveraging CJC for. CJC is releasing a blog series that covers the following:

• Expertise

• Why and What?

• How?

• Migration and Security

• Benefits and Future

In our previous post in this series, we discussed the expertise CJC has developed in public cloud vendors and technologies to assist firms to transform and migrate their capital markets platforms, products, applications and databases to the public cloud. We have discussed how we used this expertise inwards as well, after all, CJC is a global 24x7 operation running in 4 key final locations, with clients connecting into our firms from almost 30 geographical locations. We have covered how our teams reviewed this global infrastructure and selected which infrastructures were in scope for public cloud migration, namely:

• Our infrastructures which provide IT monitoring to clients globally

• Our Big Data IT analytics product mosaicOA

• Both operate from key data centres such as Equinix

Identification is one thing, however, we needed to fully test if the public cloud would 100% work. We needed to ensure:

• Would the aspects of software, not originally designed to work in the cloud migrate successfully?

• Would our real-time monitoring be affected, we are held against strict SLA and KPIs and an important third-party dependency providing operational resilience for hundreds of firms in the capital markets?

• Could the cloud handle the hundreds of thousands of updates per second that our powerful time-series databases have to handle? Would the performance be affected?

• Could we easily migrate from the existing physical/private cloud platform to the new public cloud – without any interruption or gaps in data?

• Identify any ‘known unknowns’ or ‘unknown unknowns’ – one example is egress costs, we have as much data leaving the cloud as we do going into it.

CJC Cloud Migration Methodologies

Cloud migration is the process of moving data and systems to the cloud, and it involves a ton of consideration. A few important ones are cost, security, performance parameters, interoperability, business continuity, to name a few. However, all of these are dependent on what is being moved and to what type of cloud.

Typically, organizations move data, applications, as well as a few other essential business elements to the cloud. Most often, the data is being moved from the local servers or data centers to the cloud. There could also be a cloud-to-cloud migration if you move from one cloud to another.

There is a cross-section of software throughout our private cloud, from world recognised operating systems, enterprise-class monitoring tools, open-source time-series database and our own, purpose-built analytics platform. No matter if rubber-stamped as ‘Cloud Ready’ – we needed to know if the public cloud was ready for the demands that CJC and our clients throw at it. Hundreds of thousands of updates per second across highly encrypted lines and infrastructure. All had to be tested.

From a software level, our Cloud Acceleration Team (CAT) set up a project separating every element of our in-scope systems to its constituent parts. The software development team, project managers and operations engineers would provide input into each part via internal workflow tools covering the Design, Software, Connectivity, Security, Testing, and Service. Each element would have a ‘Must Have’ or ‘Nice to Have’ comment, for example:

Almost 100 elements were detailed, which was vital for our CAT team to understand, architect and build out a test plan for a future estate.

As mosaic is an IT Analytics tool, we looked at the in-house estate to provide understanding of the system prior to transformation to public cloud. Here we could investigate a variety of behaviours from networking, CPU, memory and application statistics over weeks and months.

These behaviours were carefully reviewed, logged and used to provide cloud infrastructure specifications as part of the success criteria of QA and UAT testing.

The goals of each and every organization will differ and hence, the cloud migration strategy will depend on that. The most important of the decisions would be on what to move, and to what type of Cloud; Public, Private or Hybrid. It would be good to have a pilot migration phase, to evaluate load, performance, security, interoperability, dependency, and most of all, business continuity.

Real World Testing

The culmination of this process would create a migration plan document for every client.  This would instruct the CJC cloud engineers the following:

• Existing Infrastructure Information

• Application Inventory

• The Applications to be deployed.

• Application Dependencies

  • The OS and other prerequisites

• Supporting Services

  • 24 x 7 or less requirements

• Regions and Zones

  • Regions where infrastructure will be deployed

• Cloud Hardware Requirements

• Infrastructure Resource Hierarchy

• Security

• Identity and Access Management

  • Vital for Zero Trust setups

• Connectivity and Networking

• DNS Options

• Data Transfer

The document would come complete with extensive detail on each application's function, dependencies, specification, and networking requirements. The approved-on infrastructure costs, who can access these servers - both from a CJC and client perspective, the security plugins etc.

Only once this work is complete - the cloud engineer can begin the installation process, leveraging the documentation to know exactly how to create the project and build out the server platforms.

The products and tools would be tested by our development and operational teams to ensure that performance, stability and migration would be at the high standards CJC set for our client base.

Once we had completed testing, we were finally able to start the process of true migration.