Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

Rk9SRjN6S0RXbHh5c2M4eGRJSk42bmFDWFE9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

Ronald McDonald House

Marketing & Events Coordinator Job at Ronald McDonald House

 ...Reporting to the Director of Development, the primary responsibilities of the Marketing & Events Coordinator are to effectively communicate the mission of RMHC of Northwest Florida across diverse platforms and expand fundraising through event management. The Marketing... 

NoGigiddy

High-Paying Remote Customer Service Jobs - Starting at 19 Per Hour Job at NoGigiddy

 ...supportive role and are committed to creating exceptional customer experiences from their home office. Key Responsibilities: Address...  ...: No previous experience or academic qualifications are required. Candidates need a quiet, distraction-free workspace and reliable... 

Parkland Health and Hospital System

Physical Therapist- Inpatient Rehab Job at Parkland Health and Hospital System

 ...programs for individual patients according to the principles of physical therapy to facilitate the patients recovery and return to...  ...in Texas, or an active compact license permitting a Physical Therapist to practice in Texas. Must have a current Healthcare Provider... 

Yuanzhu Book Sales Center,Lanshan District,Linyi City

English Teacher (Online/Offline) Job at Yuanzhu Book Sales Center,Lanshan District,Linyi City

Conduct engaging English lessons tailored to students needs (beginner to advanced levels). Teach classes online or in-person, depending on your preference and availability. Create and implement lesson plans, assignments, and assessments. Utilize online teaching...

ProSidian Consulting, LLC

Oversight and Process Improvement Coordinator Job at ProSidian Consulting, LLC

 ...Management | Compliance | Business Process | IT Effectiveness | Engineering | Environmental | Sustainability | Human Capital. We help...  ...for public and private, defense and civilian government, and non-profit organizations. Our solution-centric services are deployed across...