Datafloq

Data and Technology Insights

Senior Data Engineer

Location:  Atlanta, GA, USA /

APPLYCompany: Vaco Technology
Skills required: Analytics, Architecture, Data Analytics, Data Quality, Design, Engineering, Infrastructure, Marketing, Other, SQL
Technologies required: Architecture, Google Analytics, Python, SQL
Date: August 06, 2022

POSITION SUMMARY

The Sr. Data Engineer will join the Information Management and Analytics (IMA)

team to drive the implementation of the reset data strategy, advance data architecture with the Data

lake analytics platform (DLAP) and advance data quality and master data.

The role's mandate is to work holistically across multiple business teams (Marketing,

Finance, Operations) and IT domain areas to build and maintain foundational data and analytics

infrastructure essential to turning data and information into corporate assets, driving

revenue growth and accelerating customer acquisition and retention.

n and communicate highly complex data problems into simple, You will seek answers

to business questions via hands-on exploration of data sets via MS-Azure Databricks notebooks, SQL,

dashboards, statistical analysis, data models and data visualizations.

The Senior Data Engineer will also create measurement infrastructure by working closely with our

marketing, finance and operations, analytics, partnerships, data architecture and business intelligence

teams to build best-in-class data pipelines and processes that stitch together complex sets of data

stores and drive decisions. Your work will set the foundational KPIs against how we operate and

optimize our business. As a believer in Digital Transformation and being part of the IMA team, you will

be expected to share knowledge, relentlessly problem solve, have a superior work ethic and be an

excellent communicator in a fast-paced and complex environment.

RESPONSIBILITIES:

Data engineers are tasked with managing and organizing data, while also keeping an eye out for

trends or inconsistencies that will impact business goals. It's a highly technical position, requiring

experience and skills in areas like programming, mathematics, and computer science. But data

engineers also need soft skills to communicate data trends to others in the organization and to help

the business makes use of the data it collects. Some of the most common responsibilities for a data

engineers include:

  • ‘ Strong leadership in architecture thinking in planning, defining the future state but practical

expertise to deliver in byte-size chunks

  • ‘ Own end-to-end data flow for all business data in Marketing solutions like Adobe, SaaS vendors

in operations, finance etc.

  • ‘ Collaborate with business and third-party resources to translate business

requirements into data engineering, working with data architecture to turn them into reality.

  • ‘ Configure, maintain, and support analytics solutions through developing ETL pipelines to

marketing data sources (e.g., CDP, CRM, ERP systems, Marketing Automation platforms)

  • ‘ Understand business context and translate stakeholder needs into data requirements, data model

for DLAP, data structures, ETL and dashboard requirements. Translate marketing stakeholder

requirements into technical specifications

  • ‘ Design, build and maintain critical data pipelines and dashboards to ensure highly accurate and

reliable business reporting.

  • ‘ Monitor daily execution, diagnose, and log issues, and fix business critical pipelines to ensure

SLAs are met with internal stakeholders.

  • ‘ Bridge gap between business requirements and ETL logic by troubleshooting data discrepancies

and implementing scalable solutions.

  • ‘ Make data model and ETL code improvements to improve pipeline efficiency and data quality.
  • ‘ Own data import/export pipelines and incorporate into existing workflows to enable reporting and

optimization efforts

09/29/21

Page 3

  • ‘ Good Understanding of Modern Lake house architecture preferably using MS Azure and

Databricks stack.

  • ‘ Good practical Understanding of data quality and master data management solutions and ability

to realize the mastering process within the Azure Databricks via external data API solutions like

Melissa data (Personator consumer, matchup object etc.)

  • ‘ Some expertise in ESB, Rest API and Azure events hub for data ingestion, data consumption,

services orchestration and messaging

  • ‘ Design, maintain and support system and web analytics solutions using tag management

systems and customer data platforms (e.g., Google Analytics, Adobe Analytics, Google Tag

Manager, Tealium, DDX, etc.)

  • ‘ Support Analytics Quality Assurance Testing and help develop processes for data mastering and

quality control.

  • ‘ Provide on-demand support for issues through troubleshooting, issue identification, resolution,

and reporting.

  • ‘ Monitor data infrastructure for anomalies and support rapid resolution for mission-critical

reporting.

  • ‘ Collaborate with teams across the organization to deliver innovative products that help improve

content sharing, user-to-user communication and user retention

  • ‘ Ensure we have the right data and work with the team to develop workflows to supply the data.
  • ‘ Working with Data Development lead to Architect,. Design and develop all aspects of the DLAP –

Data lake analytical platform powered by the MS Azure stack.

  • ‘ Assist in any hands-on development of translating requirements and use cases to
  • ‘ design with modular and scalable code in the MS Azure stack
  • ‘ Collaborate with various stakeholders and arrive at an optimal and balanced
  • ‘ solutions in agile/scrum project delivery cycles. Advance each sprint (2 weeks) of
  • ‘ new functional and technical capabilities in the DLAP development
  • ‘ Work with IMA data team to assist in end-to-end data flow from data pipeline,
  • ‘ data ingestion, data quality, data mastering, lake warehouse and data analytics (data science and

some ML) to all data consumption endpoints (such as Adobe, DOMO)

  • ‘ Have a position and help set up the configurations for orchestration, workflow, service bus,

business rules and data governance processes

  • ‘ Have a position and help set up the configurations for data privacy/PII data in encrypting sensitive

and confidential data

  • ‘ Design, develop and unit-test development in ADF, Azure Databricks (ADB), Azure
  • ‘ functions, Python, Azure synapse with proven SQL and data design knowledge.
  • ‘ Knowledge of Azure Logic Apps, Event Hubs may be required.
  • ‘ Document trade-offs to architecture and design with impacts to risks, costs and
  • ‘ technical debt
  • ‘ Understanding of Data Analytics – data science, regression modeling, ML using Azure Synapse,

AzureML, Databricks ML possibly.

  • ‘ Work closely with Azure Infrastructure admin to ensure compliance to
  • ‘ infrastructure and security layouts

QUALIFICATIONS:

  • ‘ Minimum of Bachelor's degree and 6-8+ years of work experience with a focus on data platforms,

data architecture and engineering and analytics in a data engineering, business intelligence, or

technical data role supporting Marketing organizations.

  • ‘ Strong Experience working on data model design, data logging, and data validation.
  • ‘ Experience in building data pipelines, data wrangling, transformations and consumptions
  • ‘ Strong working ability in defining data quality and master data rules for matching, merging and

de-duplicating data.

  • ‘ Strong business intuition and ability to understand complex business systems, data architecture,

and software design patterns.

  • ‘ Excellent communication skills, particularly when explaining technical matters to less technical coworkers.
  • ‘ Strong problem solving and creative-thinking skills.
  • ‘ Ability to break down and communicate highly complex data problems as simple, feasible

solutions.

  • ‘ Ability to communicate technical concepts to non-technical audiences.
  • ‘ Ability to translate business requirements into technical specs for developers.

Technical Skills:

  • ‘ Expertise in Any ETL tool i.e. (SSIS, Informatica, Data Stage)
  • ‘ Expertise to Implementing Data warehousing Solutions
  • ‘ Expertise MS-Azure stack (Azure Data Lake, Azure Data Factory, Azure Databricks) —

Mandatory

  • ‘ Some understanding of other Azure services like Azure Data Lake Analytics & U-SQL, Azure SQL

DW

  • ‘ Good understanding of Azure Databricks platform and can build data analytics solutions to

support the required performance & scale

  • ‘ Good Understanding of Modern Data Warehouse/Lambda Architecture, Data warehousing

concepts

  • ‘ Ability to use word to create required technical documentation like solutions design, architecture

and design diagrams, detailed comments in code base

  • ‘ Guide other developers and project team to manage team development tasks, manage

dependencies, own technical issues log and resolve open items in a timely manner

Job Requirements:
QUALIFICATIONS:
Minimum of Bachelor's degree and 6-8+ years of work experience with a focus on data platforms,
data architecture and engineering and analytics in a data engineering, business intelligence, or
technical data role supporting Marketing organizations.
Strong Experience working on data model design, data logging, and data validation.
Experience in building data pipelines, data wrangling, transformations and consumptions
Strong working ability in defining data quality and master data rules for matching, merging and
de-duplicating data.
Strong business intuition and ability to understand complex business systems, data architecture,
and software design patterns.
Excellent communication skills, particularly when explaining technical matters to less technical coworkers.
Strong problem solving and creative-thinking skills.
Ability to break down and communicate highly complex data problems as simple, feasible
solutions.
Ability to communicate technical concepts to non-technical audiences.
Ability to translate business requirements into technical specs for developers.
Technical Skills:
Expertise in Any ETL tool i.e. (SSIS, Informatica, Data Stage)
Expertise to Implementing Data warehousing Solutions
Expertise MS-Azure stack (Azure Data Lake, Azure Data Factory, Azure Databricks) —
Mandatory
Some understanding of other Azure services like Azure Data Lake Analytics & U-SQL, Azure SQL
DW
Good understanding of Azure Databricks platform and can build data analytics solutions to
support the required performance & scale
Demonstrated analytical and problem-solving skills, particularly those that apply to a big data
environment
Good Understanding of Modern Data Warehouse/Lambda Architecture, Data warehousing
concepts

Tweet
Share
Share
WhatsApp