Utilizing LLMs in OTP Applications - Sebastian Göttschkes | Code BEAM Europe 2024

CodeSync · May 9, 2025, 1:00pm

This talk was recorded at Code BEAM Europe in October 2024. If you’re curious about our upcoming event, check https://codebebeameurope.com

Adding an LLM like GPT or Gemini might seem like integrating any other REST API at first glance. But these kinds of APIs pose unique challenges! Responses might differ widely from one request to the next. Streams need specific formats for the application to be able to get all the relevant data and forward the stream to a client. Version changes between LLM models might cause the responses to change in unexpected ways. Automatic testing is error-prone and can get expensive. And what to do with those restrictive rate limits?

We will explore the general concepts of integrating LLMs into an Elixir application and then find answers to all the challenges mentioned above.

Comments welcome! View the code-sync tag for more Code Sync talks!