Position:
Full-Time
Location:
Remote/Offshore
Start Date:
ASAP
Nationality:
Any
Working Hours: EST Time Zone Windows Management Experts (WME) is looking for a motivated and experienced Senior Engineer, Global Data & Integrations Data Management to join one of our client companies. Your primary responsibility will be to build data pipelines, model and prepare data, perform complex data analysis to answer Business questions, build and automate data pipelines and quality framework to enable and promote self-service data pipelines and assist in operationalizing the AI / ML Engineering solutions.
This role is expected to lead and guide other team members and evangelize the design patterns as well as coding standards.
This role plays an active part in our Data Modernization project to migrate from on-prem platforms such as IBM Netezza to a cloud project. In this role, you’ll grow your technical expertise, apply best practices, and use tools, including Microsoft SQL, SSIS, such as SQL Server Integration Services to design databases, build data models, conduct ETL processes, design and manage data warehouse applications.
With your drive to establish processes and lead technological innovation, you are expected to make a lasting impact on our client company. That said, the candidate should demonstrate great skill in creativity and innovation, the ability to thrive in a pressure environment, and possess excellent communication skills.
Also, the successful candidate will be a self-motivated individual, who can work effectively in dynamic conditions and within deadlines. That said, the preferred candidate should expect to work 40 hours per week and should be flexible enough to work part-time hours as well.
NOTE: THE RESOURCE MUST BE OFFSHORE.
Required Skills/Qualifications/Education:
- A Bachelor’s degree in Computer Science or Engineering or Mathematics or a related field and 5+ years of experience in various cloud technologies within a large-scale organization.
- 5+ years of experience in an Enterprise Data Management or Data Engineering role.
- 3+ years of hands-on experience in building metadata-driven data pipelines using Azure Data Factory, Databricks / Spark for Cloud Datalake.
- 5+ years of hands-on experience with using one or more of the following for data analysis and wrangling Databricks, Python / PySpark, Jupiter Notebook.
- Expert level SQL knowledge on databases such as but not limited to Snowflake, Netezza, Oracle, SQL Server, MySQL, Teradata.
- 3+ years of hands-on experience in one or more big data technologies such as Cloudera Hadoop, Pivotal, Vertica, and MapR is a plus.
- Experience working in a multi-developer environment and hands-on experience in using either Azure DevOps or GitLab.
- Preferably experienced in SLA driven Production Data Pipeline or Quality support.
- Experience or strong understanding of traditional enterprise ETL platforms such as IBM Datastage, Informatica, Pentaho, Ab Initio etc.
- Functional knowledge of some of the following technologies – Terraform, Azure CLI, PowerShell, Containerization (Kubernetes, Docker) • Functional knowledge of one or more Reporting tools such as PowerBI, Tableau, OBIEE.
- Team player with excellent communication skills, ability to communicate with the customer directly and ability to explain the status of the deliverables in scrum calls.
- Ability to implement Agile methodologies and work in an Agile DevOps environment.
- Personal Attributes: Self-starter, Collaborative, Curious, Strong work ethic, highly motivated, Team oriented.
- Experience designing and building complex data pipelines in an agile environment.
- Expertise in data analysis and wrangling using SQL, python, and databricks.
- Experience with modern cloud development and design concepts; software development lifecycle; multi-developer code versioning and conflict resolution; planning, design, and problem resolution enterprise data applications/solutions.
- Demonstrated ability in developing a culture that embraces innovation, and challenges existing paradigms.
Responsibilities of the Desired Senior Engineer, Global Data & Integrations Data Management:
- Team up with the engineering teams and enterprise architecture (EA) to define standards, design patterns, accelerators, development practices, DevOps, and CI/CD automation.
- Create and maintain the data ingestion, quality testing, and audit framework.
- Conduct complex data analysis to answer the queries from Business Users or Technology team partners either directly from Analysts or stemming from one of the Reporting tools such as PowerBI, Tableau, or OBIEE.
- Build and automate the data ingestion, transformation, and aggregation pipelines using Azure Data Factory, Databricks / Spark, Snowflake, and Kafka as well as Enterprise Scheduler tools such as CA Workload automation or Control M.
- Setup and evangelize the metadata-driven approach to data pipelines to promote self-service.
- Setup and continuously improve the data quality and audit monitoring as well as alerting.
- Constantly evaluate the process automation options and collaborate with engineering as well as architecture to review the proposed design.
- Demonstrate mastery of build and release engineering principles and methodologies including source control, branch management, build and smoke testing, archiving, and retention practices.
- Adhere to and enhance and document the design principles and best practices by collaborating with Solution and in some cases Enterprise Architects.
- Participate in and support the Data Academy and Data Literacy program to train the Business Users and Technology teams on Data.
- Respond SLA-driven production data quality or pipeline issues
- Work in a fast-paced Agile/Scrum environment.
Identify and assist with the implementation of DevOps practices in support of fully automated deployments. - Document the Data Flow Diagrams, Data Models, Technical Data Mapping, and Production Support Information for Data Pipelines.
Core Values:
- Integrity (Doing What’s Right)
- Inclusion (Encouraging Diversity)
- Teamwork (Working Together)
- Excellence (Being Your Best)
- Accountability (Taking Personal Responsibility)
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status.
