Senior Data Engineer (Databricks, Azure)
The Opportunity
Step into a pivotal role as a Senior Data Engineer with a rapidly expanding cloud-based logistics platform serving over 5,500 e-commerce merchants. You'll work directly with U.S.-based product, engineering, and data science teams on a major infrastructure modernization, migrating the data platform from Azure Synapse to Databricks. This is a hands-on engineering role: you'll own table migrations end to end, build the automated validation that proves those migrations are correct, and stand up the pipelines that power the next generation of merchant-facing analytics, including benchmarking and recommendation products. This role is about owning workstreams with minimal oversight from day one. You'll be trusted to make sound architectural and tooling decisions, get into the weeds of pipeline performance and data integrity, and deliver reliable, production-grade data as the foundation everything else is built on.
What You'll Bring
- Over 5 years of hands-on data engineering experience, ideally within the Azure ecosystem.
- A concrete, personally owned data platform migration you can walk through end to end, from source systems and tooling decisions through cutover, including what broke and how you fixed it.
- Systematic, automated data validation and integrity checking on migrations: reconciliation, row and aggregate counts, schema and null checks, and automated alerting when a load is off. This is a core, day-one responsibility.
- Deep, hands-on Databricks: development and optimization of production pipelines, Delta Lake, and medallion (bronze, silver, gold) architecture.
- Hands-on Unity Catalog governance and access control, including column level and row level security and data masking, not only cataloging and lineage.
- Strong SQL and T-SQL, relational database design, data modeling, and data warehousing and ETL/ELT pipeline architecture.
- Hands-on experience with the Azure data platform, specifically Azure Data Factory and Azure Synapse Analytics.
- Professional English communication: strong communication skills, enabling effective collaboration with both technical and nontechnical stakeholders.
What Sets You Apart
Optional skills and technologies:
- Experience with Unity Catalog metric views.
- Building AI and ML solutions in production, such as NLP and LLM applications, recommendation engines, predictive models, or intelligent search and query tools. These pipelines will feed AI-powered merchant products, so exposure or genuine interest is a plus, though not required for the core work.
- Familiarity with AI and ML frameworks and tools such as LangChain, Azure OpenAI Service, Azure ML, or HuggingFace.
- Data security and access control patterns for multi-tenant or customer-facing applications.
- Familiarity with Power BI, including AI-powered features like Copilot, smart narratives, and Q&A visuals.
- Experience with AI development tools like Claude, Copilot, Cursor, Databricks Genie, or Replit for accelerated development workflows.
- Familiarity with data governance and cataloging tools such as Open Metadata.
Why You'll Love Working With Us
Join a dynamic, mission-driven environment where your work directly supports over 5,500 merchants. You'll collaborate with top-tier U.S.-based teams in a remote-first setting, owning a core part of a major infrastructure modernization from Synapse to Databricks and helping lay the data foundation for cutting-edge AI initiatives. Our tool-agnostic culture values the best solution over any specific technology, fostering a fast-paced engineering environment with real ownership, variety, and room to grow.
- Locations
- Latin America
- Remote status
- Fully Remote
About Flylance
We offer you the unique opportunity to work on cutting-edge projects that push the boundaries of technology. Imagine being part of a team that's developing the next big advancement in A.I., or crafting software solutions for Fortune 500 companies. This isn't just a job; it's your chance to make a mark on the tech landscape, showcasing your skills on projects that are at the forefront of innovation. By joining us, you're not just building software; you're building a legacy.
Check out our website to learn more.