Databricks Engineer

The Data Engineer – Databricks is responsible for assembling large, complex sets of data from various data sources using APIs. Primary responsibilities include designing and developing new data pipelines, improve performance to existing data pipelines, support and maintain data pipelines, and provide expertise and best practices regarding data bricks. Partners with other BI team members to support data infrastructure needs while assisting with data-related technical issues. Responsible for identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. Maintains all tools and processes in conformance with the organization’s policies and objectives.

• Partners with global team of BI Developers, SMEs and BI Leaders to evaluate the feasibility of and provide timetables for executing requirements to deliver data & API solutions in Agile manner.

• Translates business requirements into meaningful, clean, and scalable enterprise-grade data solutions built according to established Pontoon Business Intelligence & Analytics best practices.

• Design and build data integrations using APIs, REST APIs(JSON) and web services.Ingest, transform,and publish data from sources and to destinations via an API.Build out new API integrations to support continuing increases in data volume and complexity.

• Plan, design, develop, test, and maintain REST APIs.Propose Data load optimizations and continuously implement to improve the performance of the Data loads.

• Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.Performs data analysis required to troubleshoot data-related issues and assist in the resolution of data issues.

• Develop and maintain scalable data pipelines in data bricks to integrate with enterprise data streams. Build data pipelines that feed business intelligence tools and expose data to end users using Power-BI.

• Design and implement Data Architectures. Works closely with other team members to develop a strategy for long term data platform architecture. Responsible to develop data architecture that is modular and cloud centric for agile delivery.

• Develop POCs and other technical evaluations of technologies, designs,and solutions to influence platform architects. Drive system and platform thinking within BI development team.Assists in estimation and assessment of feasibility of features.

• Support the planning and implementation of data design services, providing sizing and configuration assistance and performing needs assessments.

• Analyze current business practices, processes, and procedures as well as identify future business opportunities for leveraging cloud services using Databricks.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Lead multiple activities including but not limited to data models, APIs, partner integrations, and cloud infrastructure.

• Bachelor’s degree in computer science, statistics, applied math, or information systems with a minimum of three years of experience as Data Engineer, API Developer, Databricks Developer/Engineer, Software Engineer, or Backend software engineer/developer.

• Three plus (3+)years of experience with extracting and modeling data from API& JSON required. • Two plus (2+)years of working experience with SQL required.

• Three plus (3+)years of experience designing, developing, validating, and automating ETL/ELT processes.

• Three plus (3+)years of experience with Databricks platform and its tools including Platform (notebooks, clusters, Jobs, Databricks SQL, relational entities, Repos) Apache Spark (PySpark, DataFrame API, basic architecture) Delta Lake (SQL-based Delta APIs, basic architecture, core functions) Databricks CLI (deploying notebook-based workflows) Databricks REST API (configure and trigger production pipelines) required.

• Three plus (3+)years of experience in Python and common python libraries required.

• Experience working with Power BI and/or other Business Intelligence visualization tools preferred.

• Experience managing multiple development projects at one time is required.

• Skilled in communicating effectively verbally and in writing.

• Ability to establish and maintain effective working relationships.

• Ability to work effectively both independently and in a group or team environment.

• Ability to prioritize, organize and perform multiple work assignments simultaneously while meeting assigned deadlines.

• Ability to understand end-user needs and assist in the development of data products. • Experience designing and building modern data pipelines and data streams.

• Experience with API (JSON) and Streaming technologies.

• Experience with Databricks (Azure Databricks REST API, Delta Tables).

• Experience with Python.

• Experience indata architecture for transformation and modernizationof enterprise data solutions using cloud data technologies.

• Advanced SQL Programming with structured and unstructured data and object-oriented/object function scripting languages (Scala).

• Strong knowledge of DataIntegration (e.g.,Streaming, Batch, Error,and Replay) and data analysis techniques.

• Experience with Security Concepts (e.g.,Encryption, Identity, etc.).

• Good problem-solving skills. Attention to detail.

• Self-directed and can stand on their own two feet and get stuff done.

• Experience with agile development methodologies.

• Proficient with Microsoft Office products such as Word, Excel, PowerPoint, and Outlook.

• Open to working in US hours ( 8am EST - 5pm EST)

It’s an exciting time to be part of our team. We’re proud to be a global thought-leader and care about doing the best job we can to ensure better futures for everyone. We do this by building our Future@Work strategy as a united team of 30,000+ colleagues with a collective spirit working in over 60 countries globally. You’ll have the opportunity to grow across a variety of interesting jobs and careers over our extensive portfolio of global brands. We empower our colleagues to work in the smartest, most efficient ways to achieve total balance between the demands of their jobs and their lives. That’s why we give you the autonomy and support you need to work in the way that makes you most productive, agile, confident, and insightful. Putting people first, pioneering with a collective spirit and always advancing with a growth mindset -that’s what we stand for at the AdeccoGroup. Here, we are all individuals with a unique perspective on the world we live in. That’s what makes us stronger. Whoever you are and whatever your background, you can be yourself. So, we aim to build on the attributes that make you, you. We offer a range of world-class resources for upskilling and development, satisfying your curiosity while the sharing skills, knowledge, and expertise to grow together. Make an impact where it matters most.

We believe that having an understanding of the hiring process helps you to prepare, feel, and be, at your best. As a global, multibrand organization with multiple different roles, our application process can vary. On our career site, you will find some of the key steps you can expect to guide you along the way. As one of the world’s largest employers we believe in talent, not labels, and focus on the diverse and unique skills our people bring. We seek to foster a culture of belonging and purpose, an environment where everyone can thrive and feel engaged, and where difference is respected and valued. Our commitment to equity, equal opportunity, inclusion, and diversity is part of our broader commitment to respecting fundamental human rights across our value chain. The Adecco Group is proud to be an Equal Opportunity Employer.

Posting date: 25-02-2024