Sometimes, you just need the right time to appreciate something and get into it properly.
In the past year, I have found myself rather disillusioned with many aspects of what I work with and what I do. So I have been looking for things that can re-ignite my motivation and joy with what I do.
Lately, I have found a few of these things.
One of these things has been Elixir and the BEAM virtual machine. (BEAM = Bodgan/Björn’s Erlang Abstract Machine)
In the past two months I have dipped my toes into the Elixir language, the tooling and its ecosystem, and I am absolutely loving it!
Once you explore Elixir and its ecosystem, you’ll notice the significant focus on improving developer productivity and satisfaction. The language itself is elegant, and the associated tooling is quite good.
In addition, the underlying BEAM virtual machine and the Erlang runtime system (OTP or Open Telecom Platform) are marvelous pieces of technology. This platform works really really well for concurrent and distributed software solutions.
BEAM and Erlang/OTP are older than Python, Java, C#, JavaScript and many other modern languages and runtimes today, yet BEAM and Erlang/OTP blow most of them out of the water in terms of resilience and effective & distributed concurrency.
Companies such as Discord, WhatsApp, Klarna, Cisco and Ericsson are all using this platform in their products and services, and I have seen numbers claiming that 90% of internet traffic passes through devices running BEAM, and also 40% of the mobile networks.
Standing on the shoulder of giants
The foundation provided by Erlang/OTP and BEAM makes Elixir stand out, but it is not limited to that.
The Elixir language itself is elegant, and the ecosystem is quite good.
The community is also active and friendly, and the language creator, José Valim, is a brilliant, friendly and enthusiastic person, which definitely contributes to the feel-good aspect of Elixir.
As a starting point to learn more about Elixir, this short documentary video is a nice introduction:
You, of course, want to have a look at the Elixir website as well.
A good place to learn the Elixir language with some hands-on exercises is The Elixir track at Exercism.
There are over 150 exercises there and an extensive learning mode that introduces various concepts in Elixir one step at a time. I have completed the learning mode myself there for Elixir, and I think one was very good. Highly recommended! You can also find other BEAM languages there as well, like Gleam, Erlang, and LFE (Lisp Flavoured Erlang).
Great things about Elixir ecosystem
A language itself is not enough, a lot of value also comes from its ecosystem.
There are a few areas of great interest in the Elixir ecosystem, I think.
Web Development
The Phoenix web framework is a great example of what can be done with Elixir, and it is the most admired web framework in Stack Overflow’s developer survey.
A part of Phoenix is also LiveView, which is a way of efficiently do server side rendering (SSR) of web pages. It is like HTMX, but uses web socket connections for the data exchange and it is quite performant.
Several years ago (2015) there was a benchmark with Phoenix that established around 2 million active websocket connections.
Machine learning and data science
The Elixir ecosystem has seen a lot of success in the machine learning space, and is establishing itself as an interesting contender to the ruling language Python.
A core effort in this space is Numerical Elixir, or Nx for short. BEAM and Elixir are not so strong with raw numerical number-crunching. Nx alleviates that with building new abstractions and processing tools to perform efficient computations on tensors, and also allows to run computations on various GPU back-ends, if chosen. It is like NumPy for Python.
There are multiple other libraries that build on top of Nx, like Axon and Bumblebee for easier processing of machine learning tasks.
In the data science space there is also Explorer, which you can use to explore and transform data in data frame.
It is a wrapper around the Rust interface for Polars, and whose interface is much inspired by dplyr in R.
It can work with several data formats and data sources.
Internet-of-things and embedded software
Another important project is Nerves, which is a platform for building and deploying embedded software solutions using Elixir and BEAM.
Given that Erlang/OTP started out as embedded software in telecom switches, it feels pretty natural fit to use Elixir here as well.
Nerves support many types of devices, including things such as Raspberry PI and BeagleBone.
Serverless - kind of
You may be familiar with the term “serverless”, which is a term that refers to a way of running software in the cloud, where you rarely need to think about server setup (but there are still servers under the hood).
You also typically get automatic scaling up and down of resources needed, based on required load.
Cloud services such as AWS Lambda and AWS Fargate are examples of computational services that are “serverless”. There are many other managed cloud services one could consider “serverless”, such as AWS DynamoDB, Amazon SQS.
If you use a service like AWS Lambda, you can get very granular control over what you can scale up and down, but at a cost of dependency on additional services, and a need to adapt the solution and architecture to the constraints of the AWS Lambda platform.
A recent project called FLAME (Fleeting Lambda Application for Modular Execution) aims to provide the benefits of Functions-as-a-service (like AWS Lambda), but without many of the drawbacks.
With FLAME, developers can run their normal application code and choose specific functions in the code to execute with Lambda-style auto-scaling. Using a FLAME code wrapper, FLAME can, behind the scenes, start up and show down servers as needed. FLAME allows setting suitable limits if desired, to keep resources and costs in check. In theory, any language can use the FLAME pattern - but it is especially suitable for Elixir, or other languages that use the Erlang/OTP platform.
Livebook - computational notebooks and more
At first glance Livebook looks like an alternative to Jupyter notebooks, but specifically for Elixir.
Livebook is a type of computational notebook like Jupyter, but it is also much more. Not only did they address many of the shortcomings and issues with Jupyter notebooks, but they also extended it to be an application builder solution. Livebook can run its own BEAM instance by default, but can also use an external BEAM instance and can connect to running instances for exploration of running systems.
One of the interesting features of Livebook is called smart cells. These are components you can include in your notebook for easier and visual processing of data with no need to write code for that yourself.
The code of the smart cells is not visible, allowing it to be used as a no-code/low-code environment in a way. If you want to, you can get the code that the Smart Cells generate, so if you have coding skills, you can get that and adjust it as you see fit.
Many of the tools and projects for both Nerves and Nx use Livebook. I am quite excited about Livebook and I think it has the potential to be a “killer app” for Elixir. I.e. people will learn Elixir in order to use Livebook to the fullest.
Next steps
Personally, I am quite excited about many of these areas mentioned above! I plan to do a lot with Livebook, and I have a few projects which I am looking forward to start as soon as I can. Elixir, its ecosystem and community have brought back the fun and joy!