Data Engineer

Overview

Responsibilities:

• Establishes database management systems, standards, guidelines and quality assurance for database deliverables, such as conceptual design, logical database, capacity planning, external data interface specification, data loading plan, data maintenance plan and security policy.
• Documents and communicates database design.
• Evaluates and installs database management systems.
• Codes complex programs and derives logical processes on technical platforms.
• Builds windows, screens and reports.
• Assists in the design of user interface and business application prototypes.
• Participates in quality assurance and develops test application code in client server environment.
• Provides expertise in devising, negotiating and defending the tables and fields provided in the database.
• Adapts business requirements, developed by modeling/development staff and systems engineers, and develops the data, database specifications, and table and element attributes for an application.
• At more experienced levels, helps to develop an understanding of client’s original data and storage mechanisms.
• Determines appropriateness of data for storage and optimum storage organization.
• Determines how tables relate to each other and how fields interact within the tables for a relational model.

Experience:

• Bachelor’s degree or higher or combination of relevant education, experience, and training in Computer Science.
• 6+ years experience in Data Engineering.
• 4+ years of experience working with Python specifically related to data processing with proficiency in Python Libraries – Pandas, NumPy, PySpark, PyOdbc, PyMsSQL, Requests, Boto3, SimpleSalesforce, Json.
• 3+ years of experience in Data Warehouse technologies – Databricks and Snowflake.
• Strong Data Engineering Fundamentals (ETL, Modelling, Lineage, Governance, Partitioning & Optimization, Migration).
• Strong Databricks-specific skills (Apache Spark, DB SQL, Delta Lake, Delta Share, Notebooks, Workflows, RBAC, Unity Catalog, Encryption & Compliance).
• Strong SQL (SQL, performance, Stored Procedures, Triggers, schema design) skills and knowledge of one of more RDBMS and NoSQL DBs like MSSQL/MySQL and DynamoDB/MongoDB/Redis.
• Cloud Platform Expertise: AWS and/or Azure.
• Experience in one or more ETL tools like Apache Airflow/AWS Glue/Azure Data Factory/Talend/Alteryx.
• Excellent knowledge of coding and architectural design patterns.
• Passion for troubleshooting, investigation and performing root-cause analysis.
• Excellent written and verbal communication skills.
• Ability to multitask in a high energy environment.
• Agile methodologies and knowledge of Git, Jenkins, GitLab, Azure DevOps and tools like Jira/Confluence.

Ideal Candidate:

• Tools like – Collibra, Hackolade.
• Migration Strategy and Tooling
• Data Migration Tools: Experience with migration tools and frameworks or custom-built solutions to automate moving data from Snowflake to Databricks.
• Testing and Validation: Ensuring data consistency and validation post-migration with testing strategies like checksums, row counts, and query performance benchmarks

The target hiring compensation range for this role is $56 to $62 an hour. Compensation is based on several factors including, but not limited to education, relevant work experience, relevant certifications, and location.

About Aquent Talent:

Aquent Talent connects the best talent in marketing, creative, and design with the world’s biggest brands.

Our eligible talent get access to amazing benefits like subsidized health, vision, and dental plans, paid sick leave, and retirement plans with a match. We also offer free online training through Aquent Gymnasium. More information on our awesome benefits! 

Aquent is an equal-opportunity employer. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We’re about creating an inclusive environment—one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.

#APP