Wanted to share our experience with using shmem in flussonic.
I’ve tried to used some memory tricks like mmap years ago and have refused, because mmap is almost useless for tasks like “read 40 Gbps from disks, convert from mp4 to DASH and serve to the internet”.
However, shmem happened to be rather useful.
We work with different hardware like SDI capture cards that can produce traffic about 4 gbit/s for single TV stream.
During long time all userspace libraries (C, C++) around such cards were integrated as NIFs into our main process streamer, but I’ve decided to extract them into external processes.
Reasons are:
- memory safety. Any bug in vendor C++ library will ruin whole streaming server
- kernel safety. If anything goes wrong with the driver, the calling process will be stuck. It means whole streaming server, not only one stream.
- reducing code complexity. We have more than 100K+ tests to run on each commit and the only way to reduce this enormous amount of tests is to split code into different parts with clean and well designed interface and separate tests.
But it is not a great idea of sending 400 megabytes per second through TCP socket even on localhost, so we’ve tested shared memory approach and it works.
shmem is not a message delivery channel, so it should be used together with socket. We use local file socket without connection and it allows restarting both sides: consumer and producer independently.