Many of you who have started to develop modern apps by using Microservices approach, have already learned that development tools, debuggers, performance monitoring and tracing lag behind the desired architecture.
Situation is even worse when it comes to functional monitoring, where your goal is to find out what is going on with your system from the functional point of view.
In this post I’ll share a couple of slides from my presentation from Debug conference which was held last month in Zagreb.
On the following picture you can find the typical layout of today’s enterprise architecture.
The most important to note are the following (the list is far from complete):
- besides SQL RDBMS, data is also stored in NoSQLformat (e.g. Redis) and other formats (e.g. Kafka) etc.
- data are not stored centrally on one place like in an older architecture style (Microservices = distributed system)
- Integration among legacy style architectures (client – server, Three Tier architecture) will bring more complexity to already complex architecture
- Hybrid Cloud architecture will add even more complexity
- Desired architecture should use free and Open Source components (to avoid vendor dependency trap)
To cover every detail will be far beyond the goal of this article (probably a book can be written to cover all).
Instead I’ll describe just a basic idea.
Taking into account all the challenges and constraints listed above, by combining two Open Source technologies Presto/Trino and Apache Superset, you can meet all the goals.
Apache Superset is well known free and Open Source Tableau BI tool alternative, contributed by AirBnB.
More you can find on the following link:
Presto/Trino on the other hand is one of the most successful Open Source projects overall.
The reason for two names for the same/similar project you can find here:
https://trino.io/blog/2020/12/27/announcing-trino.html
More about Presto/Trino you can find here:
Some of the most common Presto / Trino use cases are listed down below:
The most important art is the list of connectors (the list will expand as new popular technologies will get traction).
By taking distributed query engine which can read from many different data sources and visualization capabilities of Apache Superset, you can get immediate insights of your system from the functional point of view.
Down below you can find a couple of slides that is showing how it looks like.
Due to the many interdependencies, details of how to install properly all dependent ingredients (OS Linux version and distribution, Presto/Trino query engine, Apache Superset and corresponding Presto/Trino driver) will be left for the future.
Comments