Data Engineer

Job requirements

Key Responsibilities Logic Discovery and Requirements Analysis

Work with pricing analysts to identify custom definitions, calculations, transformations, dependencies, assumptions, edge cases, and ambiguities currently implemented in the Gold layer.

Reverse-engineer analyst-written SQL, spreadsheet logic, and ad hoc calculations to define intended business behaviour.

Translate legacy logic into clear technical requirements and migration plans.

Migration to the Silver Layer

Re-implement approved pricing definitions as governed, reusable, documented dbt models in the Silver layer.

Follow data modelling, naming, testing, version control, and code review standards.

Build maintainable, scalable, auditable models independent of undocumented analyst knowledge.

Apply dbt tests for uniqueness, not-null, accepted values, relationships, and business rules.

Parity Validation and Quality Assurance

Compare migrated Silver-layer results with existing Gold-layer outputs and resolve differences.

Validate results across representative periods, segments, boundary conditions, and known exceptions.

Confirm migrations only after parity review and pricing analyst approval.

Analyst Collaboration and Sign-Off

Partner with pricing analysts to clarify calculations, confirm intended behaviour, and align on expected results.

Lead walkthroughs and reviews; document analyst approval before retiring legacy definitions.

Communicate risks, open questions, dependencies, and decisions to technical and business stakeholders.

Documentation and Lineage

Document business meaning, calculation rules, assumptions, exceptions, source inputs, ownership, lineage, tests, and downstream consumers in dbt and Confluence.

Ensure documentation enables future engineers and analysts to maintain logic without relying on the original analyst.

Gold-Layer Cleanup and Cutover

Decommission ad hoc Gold-layer logic after the Silver replacement is validated, approved, adopted, and downstream dependencies are updated.

Verify retired logic is no longer used and operational documentation reflects the new source of truth.

Required Qualifications

5–8 years of data engineering experience on analytical data platforms.

Strong SQL skills: joins, window functions, CTEs, aggregations, conditional logic, and optimisation.

Hands-on dbt experience across models, tests, documentation, and lineage.

Strong Snowflake experience, including modelling, performance tuning, and warehouse concepts.

Experience refactoring, migrating, or modernising complex analytical logic.

Ability to untangle ad hoc SQL and spreadsheet calculations and validate outputs.

Strong understanding of data quality, reconciliation, testing, and release controls.

Skilled at working with non-engineering stakeholders to clarify requirements and validate outcomes.

Excellent communication and documentation habits.

Core Skills and Tools

SQL; Python a plus

Snowflake

dbt, dbt Cloud

Airflow

Confluence, Jira, Slack

Data modelling, quality validation, reconciliation, migration planning, stakeholder collaboration, and technical documentation.