Book review Designing Data-Intensive Applications The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
The book was by far the most useful, inside a full book I read in the last year or so. It may be because I am have been working on Nun-db (a real-time database), but I think I would have liked it just as much in the past as well. First of all, the book has been recommended to me by a friend that the opinion I respect a lot (A Ph.D. in Computer Science), the title immediately caught my attention and I have been reading it since the day I put it on my kindle.
It is a dense and complicated book. Don’t expect to read it too quickly. Some subjects require lots of thinking and sometimes reading and re-reading some other sources to understand. Nevertheless, if you are responsible for any big application that uses more than our database system, you should consider giving this book a try. It will make your next decisions much more straightforward, and you will feel safer moving on with your applications.
The first Highlight I have made about the book was: “It is impossible to reduce the probability of a fault to zero; therefore it is usually best to design fault-tolerance mechanisms that prevent faults from causing failures” I think this is one of the most know facts in technology today, yet every day we face some big important applications that trusts nothing will go wrong. Last year in consulting work, I found a bug that would make one of the data processor job stop working every time mongo runs an election because some node-js drivers would expect the connection to be alive forever, to restart this service would require manual intervention (what is terrible) but I am almost sure when the developer was writing that code, they did not consider too deep what could go wrong and did not prepare for it. The bug was on the code for years and would wakeup in the meed of nigth the operation team once in a while for over a year.
Databases failures do not happen on a daily basis, but it does happen once in a while, and if you consider that to scale, Mongo needs to re-run the election. This can be a big problem that will happen in the worst time ever (when the system is under too much load and it needs to scale the database up, for example).
This is to give a tiny idea of all this book has to offer. The way the author writes is also quite interesting and makes you want to keep reading every chapter after the other.
Influences on Nun-db
At the end of the book, I had the feeling that the author was talking to me about Nun-db, like at the end where he speculates about the future of database systems. He provokes us with “It would be very natural to extend this programming model also to allow a server to push state-change events into this client-side event pipeline.” when talking about Elm and React event modeling, both are perfect uses cases to Nun-db. I have proven more than once in code that adding a Nun-db database to an existing react/redux application requires nearly 0 code changes in the application, and the result is impressive. (Check out Tic-Tac-Toe end Todo examples)
This book has not only influenced Nun-db’s future, showing we are on the right path, but it has also already helped in the op-log and election implementations (future post here) (2 significant features I have been working on since last year). Those features will enable Nun-db to run into multiple machines cluster and make it much more reliable.
The book also left me with the impression that we can implement more complex databases system using Nun-db key-value capabilities, like: Time-series databases and graph databases (but these are things for many years from now).
Fantastic book and recommended for software architects, tech leads, CTOs, or any developer that wants to make better-informed decisions on data systems. Reading this book will pay off.
If you are working on a database, message queue, or data-driven application is probably even more helpful. Still, in this case, it will also serve to open the door to other more detailed documents and articles where you will find the information as deed as you need as it will give you a great view of the first level of the problem. The next levels you will probably need additional references. Still, it will be a great start, unquestionable.
I will re-read this book probably 2 to 3 months from now once I am done with the two current implementations I am working on Nun-db. I am sure it will then give me again insides for my next cycle of significant improvements, but for now, this read has been great so far, and I am happy to come to the end of the book, so I can spend time seeing the benefits of it in Nun-db subsequent deployments.
Where to find it
- Book review Staff Engineer Leadership Beyond the Management Track
- Book review Traction Get A Grip On Your Business
- Book review Designing Data-Intensive Applications The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
- Book review Atomic Habits
- Book review Adopting Elixir From Concept to Production