Interesting Data Gigs # 8: Senior Staff Engineer, Cloud at Redpanda

Why you need to follow these ex-Monzo Bank folks

If you are not a subscriber of Interesting Data Gigs, join 1370 other Data geeks that receive it directly in their inbox each week — it’s free.

🚨 Join the Interesting Data Gigs Talent Network 🚨

It’s the perfect time to be part of The Interesting Data Gigs Talent Network, where you will find amazing Data Analytics jobs from companies like Netflix, Apple, Consensys, and many more.

Let’s change the game together: Instead of people applying to companies, companies will pitch to you, so don’t wait any other moment and join today.

Hi Data Geeks.

The last week was a very rough one: My mom was very sick with a lot of vomit caused by a very bad maldigestion. So, I didn’t have the right state of mind to write to all you as you truly deserve.

Fortunately, she is OK now and makes improvements in her health; thanks to two amazing doctors who are dear friends of my family: Thank you, Yaimara and Yanelis. You are truly awesome.

Let’s focus on today’s position: it’s called Senior Staff Engineer, Cloud at Redpanda, and let’s chat about it.

But, wait: What is Redpanda and what are they building today?

Let’s explain why I’m excited about this company and this team.

First, they are building a business where the source code is open, which from my perspective, it’s one of the strongest points why you should join the company. But, what is Redpanda?

And what is this?

This is a very good description of it:

Redpanda is a streaming platform for mission-critical workloads. Kafka® compatible, No Zookeeper®, no JVM, and no code changes required. Use all your favorite open source tooling - 10x faster.

We are building a real-time streaming engine for modern applications - from the enterprise to the solo dev prototyping a react application on her laptop. We go beyond the Kafka protocol, into the future of streaming with inline WASM transforms and geo-replicated hierarchical storage. A new platform that scales with you from the smallest projects to petabytes of data distributed across the globe.

Let’s discuss some of Redpanda's features in detail:

  • The core binary is developed in modern C++, which means one thing and one thing only: It’s insanely fast

  • It’s compatible with Kafka’s API and Protocol, which means that if your company is using Kafka today, you don’t have to replace anything in your code base.

  • No Zookeeper. No JVM: if you have installed or deployed Kafka in production, you already know that managing Zookeeper is a pain in the ass. With Redpanda, you won’t have that headache because you won’t need any additional dependency on it

  • It has one of my favorite features of platforms of this kind: it has a developer-friendly CLI with an amazing feature called autotune, which allows the cluster to make low-level changes in your operating system to increase the performance of Redpanda.

  • Its Engineering crew is very technical with a lot of real-world experience managing distributed systems. So, if you join the company, I can sure you that you will learn a ton. Read more about the Redpanda employees in this series of posts.

So, let’s see some ideas about how to stand out from your job application here.

Let’s discuss some ideas on how to approach this job application (THE REAL MEAT)

If you read very carefully the job description, you will see some key things you will need for this role:

This is a role that your knowledge will be challenged every day (in a good way), and this is a unique opportunity to build a Cloud-based SaaS platform from scratch.

Funded by premier investors including GV and Lightspeed, Redpanda is building the streaming data platform for developers. We’re evolving streaming beyond the Apache Kafka® protocol into a unified “engine of record” that delivers a categorical reduction in complexity, wicked-fast performance, onboard Wasm transforms, and transparent tiered storage that gives consumers access to both real-time and historical data from a single API.

Redpanda is very well funded, and in February 2022 it raised its Series B of $50 Million led by GV along with Lightspeed and Haystack; so the total amount of cash raised is $65.5 Million if we take into account the first capital raise in 2021.

Alexander Gallego, the founder, and CEO of the company wrote this about Redpanda entering the Modern Data Ecosystem:

Modern applications are in the middle of a real-time renaissance. Whether you are improving the trading performance of your application by 100x — as showcased by Alpaca — or you are consuming your OLTP as a CDC feed into your OLAP system, Redpanda is at the center of this market evolution. Redpanda is the glue connecting our entire partner ecosystem with MongoDB, SingleStore, Materialize, dbt Labs, Deephaven, Lightbend, and tens of other data products that cache, index, store and materialize streaming data into final projections.

So, this is truly an exceptional time to be part of this seasoned crew.

Some tips?

Join the Redpanda community on Slack

It’s simply an amazing way to start. Why? Because you will be giving yourself a lot of visibility with the Redpanda team, especially if you help people on it.

I strongly believe that this is one of the best ways to build a company these days: fostering a community is an incredible path to building engagement with real users of your product, especially in the Open Source world.

Everyone from Redpanda is very active there, even Alexander. I have the probe here:

I made a question about if the team had tested Redpanda with some AWS-Graviton3 based instances, and Alexander quickly answered with this fantastic article written by Travis Downs about the topic.

Play with the product

Please, even before the application, play with the product, take some notes, and if it’s possible, clone the Redpanda Helm Chart repository (more on that later) and start making some contributions there.

The keyword here? VISIBILITY

Read about Raft and Seastar framework

Like I said before, Raft is the consensus algorithm used by Redpanda, but they made their own implementation of it. So make sure to read that.

In the case of Seastar framework, it is the advanced, open-source C++ framework for high-performance server applications on modern hardware used by Redpanda.

Seastar was started by ScyllaDB, and it has an outstanding pack of features, perfect for Redpanda:

  • Shared-nothing design: Seastar uses a shared-nothing model that shards all requests onto individual cores.

  • High-performance networking: Seastar offers a choice of network stack, including conventional Linux networking for ease of development, DPDK for fast user-space networking on Linux, and native networking on OSv.

  • Futures and promises: An advanced new model for concurrent applications that offers C++ programmers both high performance and the ability to create comprehensible, testable high-quality code.

  • Message passing: A design for sharing information between CPU cores without time-consuming locking

And a big final tip: Revisit all your core knowledge about Kubernetes, Cloud Computing, Distributed Systems, etc

Again: this will be not a simple job, this is a truly challenging role where not only your Cloud Computing, Distributed Systems, and Kubernetes knowledge will be challenged every single day; you will need to bring something remarkable to the table.

So, make sure to bring some notes to the interview after doing all this.

Believe me: your input will be heard here.

Some resources to help you to hone on this:

And of course: don’t forget to check out some of the posts in the blog related to Redpanda’s culture. There are some gems out there.

My favorite ones?

These posts are highly recommended as well because, in all of them, they shared their own motivations to join Redpanda. And these insights could help you to build rapport once you are in the interview process.

Good luck with your application, and please send me your feedback with a message through the Slack Redpanda community.

🚨 Join the Interesting Data Gigs Talent Network 🚨

It’s the perfect time to be part of The Interesting Data Gigs Talent Network, where you will find amazing Data Analytics jobs from companies like Netflix, Apple, Consensys, and many more.

Let’s change the game together: Instead of people applying to companies, companies will pitch to you, so don’t wait any other moment and join today.

Other featured jobs of the Talent Network

Why you need to follow the work of these ex-Monzo Bank guys: Luke Singham and Stephen Whitworth 

There are so many good things happening inside Monzo Bank from the engineering perspective, that many people coming out from the company; are building companies and solving problems they previously had inside Monzo.

I love to follow this kind of company because this means they are solving a problem for themselves first.

Two of the best examples are Luke and Stephen here.

In the case of Luke, he and his co-founder Matthew Sutton were accepted at Y-Combinator (YC S22):

I had the pleasure to see a demo provided by Luke, and it’s mind-blowing.

So, if you are doing any kind of Data Quality efforts inside your company, you must chat with Luke and Matthew. Seriously.

In the case of Stephen, he is the co-founder and CEO of incident.io, which is solving a very interesting problem:

incident.io is a Slack-powered incident management platform. We help high-growth companies declare, collaborate, communicate around and learn from events that disturb their normal course of business - from critical infrastructure being down, to data breaches and security incidents.

And today, they announced its Series A raise, led by Index Ventures, with participation from Point Nine (two of my favorite VC firms out there, especially in the SaaS world), Mike Krieger (co-founder and CTO of Instagram), Lachy Groom, and the Chainsmoker’s Mantis VC.

They have aligned a very good group of high-growth companies like Ramp (an incredible partner of mine), Linear, dbt Labs, Pipe, Vanta, and many more; which sends a big message to companies in the same industry out there:

I’m coming.

Then, when you read the article about the fundraising at Index from Carlos Gonzalez-Cardenas, you understand why incident.io is building something remarkable:

Stopping things going wrong is an impossible task. It’s something I experienced first-hand running operations and product at GoCardless and Skyscanner, which is why I was knocked back by the power of incident.io: having the system they’ve developed at hand would have been a game-changer.

Essentially, having a purely preventive approach to company incidents is like pushing a zero covid policy: it doesn’t work. Cases always get through, and all that’s achieved is long-lasting damage to the organization. Instead, companies need the tools to deal with problems in real-time, and learn from them. The focus must be on building resilience – something we see leadership teams and boards increasingly shifting their mindset towards.

That’s where incident.io comes in. The team has built a category-defining product which takes the chaos, confusion and emotion out of dealing with incidents, enabling all the types of problems technology-powered companies can face to be effectively resolved, even when the stakes are high.

And their users are obsessed: I’ve spoken to many in recent months that couldn’t speak more highly of the product, the team and their vision. It’s just one of the reasons we believe that incident.io have the tech, the people, the ambition and the rigor to not only create a new category, but to dominate it.

So, if this excites you, Stephen and his crew are looking for a lot of engineers, especially after this fundraising event.

Reply to this email if you reading this from Substack or simply add me on LinkedIn if you are reading this on LinkedIn.

I can send your details to Stephen directly if you want.

Interesting Open Source projects related to Data

  • Redpanda Console: is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging. Read more about its installation here https://cloudhut.dev/docs/installation

  • Redpanda Helm Chart: it is the Helm Chart for Redpanda, which provides the ability to set up a multi-node Redpanda cluster

Final words

If you’re finding this newsletter valuable, consider sharing it with friends, or subscribing if you haven’t already.

Thanks a lot for reading and I would love to hear your feedback about it.

Marcos out.

Join the conversation

or to participate.