Senior Data Engineer

Job Description:

A successful candidate should come from a strong data engineering background, is self-directed and comfortable supporting the data needs of multiple teams, systems and products. You need to have experience with NoSQL and SQL data structures and be able to analyze/transform the data using various tools. Your analytical skills and knowledge of schema metadata will be essential. The right candidate will be excited by the prospect of optimizing our company’s data architecture to support our next generation of products and data initiatives.
Bachelor’s degree or higher in an engineering or technical field such as Computer Science, Physics, Mathematics, Statistics, Engineering, Business Administration, or similar or equivalent combination of education and experience

4+ Years experience in a data engineering role supporting production systems
2+ years experience deploying and managing a production Kafka instance (managed or otherwise)
1+ years experience manipulating data using Python (experience with Pandas is a plus)
1+ years experience extracting data from REST APIs
1+ years experience managing a codebase in GitHub
Previous experience developing ETL pipelines using technologies such as Airflow (preferable), Luigi, Oozie, Azkaban, ect.
Experience manipulating and de-normalizing data in JSON format for storage in relational databases
Experience with Google Cloud Platform or AWS cloud services
(Preferred) Knowledge and experience with Kubernetes and/or Docker
(Preferred) Advanced knowledge of SQL and experience working with relational databases. BigQuery experience is an extra plus
(Preferred) Previous experience developing data models to support a data warehouse
What You’ll Do
Assist with the implementation of new systems and updates to existing systems from a data integrity and use perspective
Develop appropriate data schemas and structures for use in downstream models/reports.
Establish best practices in our data engineering practice and strategy.
Develop data management and oversight programs spanning dozens of source systems across all departments, creating new ETL pipelines and maintenance of existing ones.
Expand and optimize our data and data pipeline architecture
Evangelist of programming best practices within the Analytics team and lead by example on topics such as code clarity, judicious use of whitespace, and unit tests.
Build new ETL pipelines to warehouse new data sources as they are identified and ensure the quality of warehoused data.
Engineer capacity and performance in addition to providing forecasting and future planning as well as review and consideration of technology trends.
Recommend and develop changes to source data structures/systems based on observations of data within the context of operational use.
Assemble large, complex data models to meet the needs of operational and strategic stakeholders.
Work closely with our in-house analysts to integrate SQL data models to a dependency tree.
Maintain user permissions for Warehouse data sources and assist in user access training.
Other duties and responsibilities as assigned by management.