What we’re looking for
We are looking for a seasoned Staff Data Engineer with amazing problem-solving abilities and a growth mindset to join a team of highly skilled data engineers and data architects to build and manage the end-to-end data pipelines (batch and near real-time) using modern cloud technologies. This is a role in the central data organization at SurveyMonkey that provides actionable insights to all key business functions of the organization
As a Staff Data Engineer, you will lead data engineering initiatives and build end-to-end Analytical solutions that are highly available, scalable, stable, secure, and cost-effective. You will be reporting to the Director of Data Engineering
What you’ll be working on
- Design, architect and build data pipelines to support existing data models
- Manage the Data Infrastructure & Platform
- Data quality: Build quality checks in the end-to-end data pipelines
- Design & Build new Data models (Fact vs Dimension). Write performant/idempotent transformations in Snowflake using dbt
- Build data pipeline using Python scripting (in a modular/loop context) Write well-tested, production-ready code in Python and Snowflake SQL
- Hands-on experience implementing ETL (or ELT) best practices
- Translate business requirements, to technical specifications, form project scope, and deliver deployable code.
- Write complex data engineering Snowflake - SQL jobs that perform sophisticated queries on the entirety of our datasets
- Document our systems for internal and external stakeholders
- Monitor and debug data pipelines running on Airflow
- Participate in code reviews
- Mentor data engineers on best practices and coding standards
- Lead and collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders.
- Identifying opportunities for process improvements, automation, and optimization of data pipelines.
- Implementing innovative solutions and exploring new technologies to address complex data challenges
We’d love to hear from people with
- 8+ years experience in data engineering and Data warehousing technologies
- 4+ years experience in Snowflake/ETL/ELT or similar technologies like Redshift
- Experience building and scaling a Data Platform.
- Experience doing Proof of concept for new solutions and technologies
- Experience with AWS cloud services: S3, EC2, RDS, Spark, EMR etc
- Experience with object-oriented/object function scripting languages: Python (preferred), Java, Scala, etc.
- Experience in orchestrating, automating, and deploying production data pipelines using Airflow/Luigi, etc
- Extensive Experience with DevOps: Git, Github actions, CI/CD pipelines, Terraform, etc
- Experience with tools such as DBT or other similar technologies
- Experience with transforming, and developing data structures, metadata, dependency, and data workflows to support an Analytics function
- Experience with Designing Scalable Data Models in a Cloud Data warehouse
- In-depth knowledge of Data lakes, EDW concepts, and data modeling (Star, Snowflake, and Galaxy schemas)
- Experience in implementing data governance practices and ensuring compliance with data privacy regulations (such as GDPR, CCPA).
- Commitment to continuous learning and professional development
#LI-remote