As a Senior Data Engineer, you will own all things related to data engineering including building and maintaining data models, integrations, and pipelines for our AI software platform. This role is very hands-on and requires a structured mindset and solid implementation skills.
Responsibilities
- Build data pipelines, data processing tools and scripts
- Perform tasks such as writing scripts, web scraping, getting data from APIs etc, and integrate data from several data sources
- Automate data pipelines using scheduling tools like Airflow
- Reinvent prototypes to create production-ready data flows
- Be responsible for the technical solution design, lead the technical architecture and implementation of data acquisition and integration projects, both batch and real time. Define the overall solution architecture needed to implement a layered data stack that ensures a high level of data quality and timely insights.
- Participate in the development of documentation, technical procedures and user support guides
- Articulate benefit analysis of technologies in open source and proprietary products. Pilot and choose optimal tools and solutions
- Support data science research by designing, developing, and maintaining all parts of the data pipeline
How you will lead
- Communicate with stakeholders to clarify requirements. Craft technical solutions and assemble design artifacts (functional design documents, data flow diagrams, data models, etc.).
- Serve the team as a subject matter expert & mentor for ETL design.
- Proactively identify performance & data quality problems and drive the team to remediate them. Advocate architectural and code improvements to the team to improve execution speed and reliability.
- Harness operational excellence & continuous improvement with a can do leadership demeanor.
- Be prepared for changes in business direction and understand when to adjust designs.
- Work effectively in an unstructured and fast-paced environment both independently and in a team setting, with a high degree of self-management with clear communication and commitment to delivery timelines.
You already have
- A BS/MS degree in Computer Science, Engineering, Mathematics, Physics, or equivalent/related degree.
- Built programmatic ETL pipelines with SQL-based technologies and platforms.
- Solid understanding of databases, and are working with sophisticated datasets.
- Shown strong problem solving with acute attention to detail and ability to meet tight deadlines and project plans.
- Shown the ability to research, analyze, interpret, and produce accurate results within reasonable turnaround times with an iterative mentality with rapid prototyping designs.
- Shown technical leadership with an emphasis on the data lake, data warehouse solutions, business intelligence, big data analytics, enterprise-scale custom data products.
- Knowledge of data modeling techniques and high-volume ETL/ELT design.
- Experience with version control systems (Github, Subversion) and deployment tools (e.g. continuous integration) required.
- Experience working with Public Cloud platforms like GPC, AWS, or Snowflake.
- Familiarity with scrum/agile project management methodologies and SDLC stages.
- At least 3 years of expert experience with SQL.
- At least 3 years of experience with the AWS ecosystem.
- At least 2 years of Python development experience using Pandas.
Why work with us?
- Work closely with core team building company and product strategy
- Stimulating environment with the opportunity to work with industry's leading minds
- Competitive compensation & perks
- A culture that you would love to work in
- Several professional and financial opportunities
- Medical, unlimited PTO, and work remotely
Turing Labs Inc.
