Lead Software Engineer, Data Pipeline
At Klaviyo, we love tackling tough engineering problems and look for engineers who specialize in certain areas but are passionate about building, owning & scaling features end to end from scratch and breaking through any obstacle or technical challenge in their way. We push each other to move out of our comfort zone, learn new technologies and work hard to ensure each day is better than the last.
Klaviyo operates a real-time data analytics platform coded primarily in Python that is built for massive scale and hosted on Amazon Web Services (AWS). The Event Pipeline team ingests billions of interactions per day to power core, Klaviyo functionality and generate data-driven insights. These interactions are ingested from disparate sources and efficiently normalized, filtered, validated and enriched prior to persistence.
As a Lead Engineer on the Events Pipeline Team, you will have ownership over defining the evolutionary, technical vision for our data ingestion pipeline. You will be responsible for the design, implementation, and tuning of highly visible mission critical stream processing systems. You will be constantly challenged to improve these systems from every angle, such as monitoring, failure recovery, data quality, tooling, and much more.
Team Tech Stack:
- RabbitMQ, Kafka
How You'll Make a Difference
- Responsible for coaching engineers, managing/reviewing technical documentation and articulating a phased approach to achieving the team's overall technical vision.
- Wholly owning features that 50,000+ customers rely on daily to reach 4B+ consumers to drive their business forward.
- Working on crafting our highly scalable real-time data pipeline that processes billions of events weekly across data sets growing to petabyte scale.
- Working on a deep analytics system that provides insight into hundreds of terabytes of data.
- Be responsible for the mission, technical direction and operations of projects that have immediate impact.
- Helping architect and design robust and high performance data processing and storage systems leveraging new databases to serve different internal and external use cases. Contribute to open source data processing technologies.
- Contributing to the company as a subject matter expert in multiple areas, constantly pushing yourself to be a better engineer and to level up all of your peers within your team and within Klaviyo.
Who You Are
- 10+ years of experience in software development, built and delivered at least one complex data processing system
- 2 years of experience in a people management role or team leadership role, coordinating the delivery of nontrivial projects.
- 5 years of experience testing and launching software products
- Experience working with reliable, fault-tolerant, and highly performant distributed systems.
- Experience with high throughput orchestration tools like temporal or cadence.
Get to Know Klaviyo
We’re Klaviyo (pronounced clay-vee-oh). We empower creators to own their destiny by making first-party data accessible and actionable like never before. We see limitless potential for the technology we’re developing to nurture personalized experiences in ecommerce and beyond. To reach our goals, we need our own crew of remarkable creators—ambitious and collaborative teammates who stay focused on our north star: delighting our customers. If you’re ready to do the best work of your career, where you’ll be welcomed as your whole self from day one and supported with generous benefits, we hope you’ll join us.
Upon request, you can receive additional information about the compensation and benefits for this role. Requests can be submitted here. Additional information regarding benefits can be found at klaviyorewards.com.
Klaviyo is committed to a policy of equal opportunity and non-discrimination. We do not discriminate on the basis of race, ethnicity, citizenship, national origin, color, religion or religious creed, age, sex (including pregnancy), gender identity, sexual orientation, physical or mental disability, veteran or active military status, marital status, criminal record, genetics, retaliation, sexual harassment or any other characteristic protected by applicable law.
You can find our Job Applicant Privacy Notice here.