8
u/rmc0d3r May 07 '20 edited May 07 '20
Rather than cherry-picking technologies like this, it would be get a good grip on fundamentals of distributed systems by reading a book like Designing Data Intensive Applications (as someone already mentioned) and then based on the problem you are trying to solve, looking up tools that do it best.
Also, there does not seem to be an Apache Kinesis; only Amazon Kinesis which is an AWS service.
And the roadmap seems to be missing mention of many commonly used database technologies, familiarity with which is essential for a big data engineer.
0
u/tynej May 07 '20
Kinesis is just rebranded Kafka isn't it?
1
u/rmc0d3r May 08 '20
AWS has another managed Kafka service, called Amazon MSK. Whereas Kinesis is in fact a group of services like Kinesis Video Streams, Kinesis Data Streams, Kinesis Data Analytics etc. So I doubt Kinesis is just a rebranded Kafka. There is no mention of such a connection in the docs as well.
1
u/tynej May 08 '20
I asked same question on some conference a guy from AWS and he just noded. I still dont know what exact part is the same but it won't be the first product which amazon claims they made on their own.
3
1
1
u/Palaquepariu May 07 '20
I think you meant Amazon Kinesis there which runs Apache Flink as a streaming framework. Also Designing Data Intensive Applications like the others mentioned is, I believe, 100% required.
You should also add some shout out to material on NoSQL distributed databases like Cassandra, MongoDB, Redis etc
Edit: typo
10
u/dubven May 07 '20
How is it missing Designing Data Intensive Applications.