At the 10th annual Flink Forward conference taking place in Berlin last week (23^rd – 24^th October), established cloud provider, Ververica, announced a new BYOC deployment of their data streaming platform, promising their customers both more flexibility and more security.

And so, with an already established foothold in the data processing industry, Ververica now offers its customers an even more convenient way to support real time data processing.

This addresses an emerging challenge for many companies today, with data processing workloads becoming increasingly complex thanks to the GenAI boom which has not only spiked the demand for real time data, but also triggered ethical concerns over issues such as data provenance, cybersecurity, and data sovereignty.

Why BYOC?

Just like the better known acronym, BYOB, BYOC is essentially all about meeting customer requirements in the most flexible and cost-effective way possible.

Alongside BYOC, Ververica continues to offer customers the two previous deployment options it provides for its data streaming services. These are:

1. A self-managed platform, where customers can host and manage Ververica clusters on-prem or in their own private cloud environment.
2. A managed services cloud, where clusters are managed and hosted on Ververica’s public cloud.

But these options are no longer meeting the more complex requirements of increasingly data-oriented AI workflows – hence the introduction BYOC.

BYOC lands somewhere in the middle of the two existing options above, giving customers the option to run Ververica’s managed streaming services in their own virtual private cloud (VPC). This extends the convenience of managed services to the security of the private cloud, which, in the words of Igor Kersic, Product Manager at Ververica, offers customers ‘the best of both worlds’.

For further convenience, the BYOC deployment can be seamlessly integrated with AWS cloud infrastructure to ensure that customers have minimal disruption to their existing set up – this compatibility will also soon be extended to other major cloud providers including Google and Microsoft.

Additionally, as the shift towards data-hungry AI workloads continues to hike up the cost of streaming via public cloud services, BYOC emerges as a cost-effective solution for businesses looking to save on networking costs.

For Ververica, BYOC is also a way to align their product with the Zero Trust Principle, a security model based on the idea that user devices should never be trusted by default, and should always be verified before use. By bringing both data and compute under one VPC that is accessed only by the customer, BYOC essentially eliminates the need for user verification while upholding the trust standards set out by this principle.

Thus, BYOC helps to ease some of major concerns emerging around the transition to AI workflows by ensuring that their customers have full sovereignty over their entire data streaming pipeline. Fundamentally, this enables businesses to securely stream real time data at scale while evading the high cost of networking in public clouds.

Why is real-time data important?

You might’ve heard the phrase ‘data is the new oil’, highlighting the emerging reality that data is becoming one of the most valuable commodities of the modern world.

Indeed, it is certainly true that without data, AI applications have very little tangible value other than being a shiny new tool for businesses to brag about with a nice range of theoretical use-cases.

So if AI is the new engine for business growth, data is most certainly the new oil. But just as actual oil varies in quality and must refined for different purposes, data, too, comes with variation in terms of where it comes from, how it is processed, how it can be used, etc.

And today, with businesses competing to get the maximum ROI from their digital assets, there is one form of data that has become particularly valuable: real time data.

But what does real time data mean exactly? After all, as Mike Gualtieri, Forrester Analyst and guest speaker at Flink Forward, points out, all data originates in real time.

“A lot of clients will say, ‘well, I don’t have any real time use cases or real time data’, but actually, every company has real time data because all data originates in real time. So what they’re really saying is that they’re not using the real time data.” ~ Mike Gualtieri, Analyst at Forrester

Nevertheless, when it comes to the role of data in business intelligence, real time data takes on a slightly more nuanced meaning related to how immediately the data is used to generate operational insights.

Indeed, real time data analytics differ from other types of data analytics in that they are constantly being updated via a continuous and never-ending stream of new data. This means that they are consistently up-to-date and time-specific, making them particularly useful for monitoring digital assets, or tracking the performance of social content, for example.

As metrics such as web traffic and follower engagement on social platforms become increasingly important markers of success for businesses, the demand for real time analytics will only increase. This is why analysts such as Gualtieri consider real time data to be the next big area of growth in business intelligence.

“When it comes to performance and operational insights, companies have already invested, collectively, billions and billions of dollars on things like data warehouses, business intelligence (BI) tools, and that sort of thing. But we think that where they are investing now is in real time insights. We think this is a huge growth area. Every time we look at the streaming data platforms and the reports that we produce, the readership amongst our clients keeps getting higher – and that’s because they’re very interested in these real time insights.” ~ Mike Gualtieri, Analyst at Forrester

Hannah Algar