Enchant Your AI and APIs with eBPF Magic 🪄

The presentation explores the capabilities of eBPF (Extended Berkeley Packet Filter) in enhancing AI and API functionalities. The session focuses on integrating eBPF within Kubernetes to improve the observability and control over AI applications without altering the existing code.

• Introduction to eBPF: eBPF is a technology embedded in the Linux kernel, enabling applications to run without modification while allowing developers to add features like encryption and traffic pattern changes transparently.
• Application in AI: The talk demonstrates how to use eBPF to manipulate AI application behaviors dynamically, such as live changes to prompts or models to enhance safety and efficiency.
• Infrastructure Setup: Steps are outlined for building an AI gateway within Kubernetes that intercepts and modifies traffic transparently, using a proxy and eBPF programs for monitoring and controlling AI agent interactions.
• Technical Aspects: eBPF programs are attached at various levels of the Linux kernel, like network cards and sockets, to control and observe traffic without impacting the underlying code execution.
• Security and Observability: eBPF provides robust tools for enforcing security constraints and observing application behavior across different system areas, ensuring secure and efficient operations.

Conclusion: The session concludes with the potential of eBPF in transforming application and infrastructure management by seamlessly integrating advanced features and maintaining control over AI interactions within Kubernetes environments.

This is the end of the AI-generated content.

It is a common occurrence to see applications thrown over the fence, landing somewhere in production without a second thought about their lifecycle or how they may need maintaining in the future to connect to more efficient API endpoints. Some of these applications are quickly thrown up to provide a temporary solution, but we all know that there is nothing more permanent than temporary. But day two, we may need to enable encryption, modify their API calls/behaviours, and understand and observe what these applications are doing.

In this talk we will utilise the power of Open Source, eBPF  and some of the powers of Kubernetes, to demonstrate that we can now add features to existing application behaviour both *live* and without making any changes to the application itself. From transparently adding encryption to an existing application without breaking its traffic patterns, to changing an AI application by live changing prompts or models providing safety or efficiency in application behaviour and finally providing understanding into what these applications are doing.