August 16, 2023
Cloud Multicast Performance Testing: Market Data via AWS Transit Gateway via Kafka Based Delivery

Register to receive insights from CJC

Get notified of the latest news, insights, and upcoming industry events.

Executive Summary

Building on a previous "Multicast in the Cloud" insight article, this technical document has been written to present the findings from the CJC Pub/Sub AWS Transit Gateway Feasibility POC Testing market data via AWS Transit Gateway leveraging PCAP recording and Kafka delivery. It provides high-level background to the tests, a results summary, and a breakdown of the key tests and how they were run.


Amazon MSK (Managed Streaming for Apache Kafka) is a service within Amazon AWS, it is described as a fully managed service in that it manages the installation, upgrade and failure recovery of a Kafka-based solution, based on parameters provided by the administrator. Unlike Google Pub/Sub it does not provide horizontal scaling, or decisions on the capacity i.e., the level of hardware (or partitioning) needed to service the required load or routing of data. At the end of April 2022, Amazon also released a Serverless version of MSK, allowing a system to be run without the administrator managing and scaling the cluster capacity.

The tests in this document all refer to the standard (non-serverless) MSK.

Amazon AWS Transit Gateway

AWS Transit Gateway acts as a cloud router, it can be used to connect on-premises networks and solve peering problems. But an additional feature is that it supports routing multicast traffic between subnets of attached VPCs, and to the individual server instances within a VPC.

As part of this work, an additional test was done to provide an indicative impact on latency if the Transit Gateway was used for multicast.

What's Inside? 

  • Kafka POC Feasibility Tests and Targets.
  • Kafka Base Unitary Tests.
  • 250 Bytes Target 7000 Packets Over 100-Millisecond Period.
  • 250 Bytes Target 7000 Packets Over 1 Second Period.
  • PCAP - Target 2000 Packets Over 10-Millisecond Period.
  • PCAP - Target 7000 Packets Over 100-Millisecond Period.
  • PCAP - Target 16000 Packets Over 1 Second Period.
  • PCAP - Target 9.6 Million Packets Published from the Source Over a 10-Minute Period.
  • Kafka Typical Dat Test - Replayed from PCAP - 19,268,676 Packets.
  • Overview.
  • Multicast Latency Test Results - Withing Same Zone.
  • Multicast Latency Test Results - Across Zones.


“CJC has always maintained a unique position of vendor neutrality. We do not provide data nor are we tied to any specific technology or platform. The benefit of the cloud is that we have the same access to these systems as the clients. We test impartially, using real-world scenarios based on our experience and expertise and simply report the results as we observe them. We encourage discussion and collaboration with all capital markets participants and cloud providers to provide transparency and awareness of what can be done”.

Peter Williams, Chief Technology Officer at CJC.

About CJC:

CJC is the leading market data technology consultancy and service provider for global financial markets. CJC provides multi-award-winning consultancy, managed services, cloud solutions, observability, and professional commercial management services for mission-critical market data systems. CJC is ISO 27001 certified, enabling CJC’s partners the freedom to focus on their core business. 

Connect with us on social media: LinkedIn | Twitter

Download now!


Get In Touch

Get in touch with our experts to learn how we can help you optimize
your market data ecosystem!
Arrange a Meeting