Using FLAME and Oban for ETL - Christopher Grainger | Code BEAM Europe 2024

[CodeSync]

Using FLAME and Oban for ETL - Christopher Grainger | Code BEAM Europe 2024

Using FLAME and Oban for ETL - Christopher Grainger | Code BEAM Europe 2024

This talk was recorded at Code BEAM Europe in October 2024. If you’re curious about our upcoming event, check https://codebebeameurope.com

FLAME (Fleeting Lambda Application for Modular Execution) is a new approach to Lambda architectures, copying your entire app for fast and easy execution on short-lived architecture.

At Amplified, we’ve migrated our ETL (extract, transform, load) from multiple apps across multiple clouds using RabbitMQ as an event bus to go all in on a monolithic architecture using FLAME and Oban.

Consolidating into a monolith using this combination has permitted us to dramatically simplify our codebase, making it easier to leverage the built-in conveniences of Phoenix and LiveView like Phoenix.PubSub. This gave us the freedom and confidence to build ETL-driven features that integrate tightly with our SaaS app, including real-time updates for our users.

This talk will focus on the transition from building infrastructure for the BEAM towards building infrastructure with the BEAM. FLAME offers a new paradigm for ‘BEAMOps’, and this talk will highlight emergent patterns and benefits.

Comments welcome! View the code-sync tag for more Code Sync talks!