IT Industry News. Daily.
Data Engineering, Analytics, Mathematics, or Statistics or Matric with 6-8 years of experience in data engineering, database development, or data management in production environments. Proven hands-on experience with SQL Server, including advanced T-SQL development, ETL/ELT workflow design, and performance tuning.
Demonstrated delivery of production data solutions—both batch and near real-time—within enterprise environments.Track record of implementing data governance and quality frameworks, ensuring compliance and traceability. Experience in orchestrating complex data pipelines using SQL Server Agent, SSIS, Airflow, or Azure Data Factory.Exposure to Power BI or equivalent visualization tools for reporting and analytics enablement.Key Skills and Competencies: Advanced SQL Server Development: Strong proficiency in T-SQL, stored procedure design, query optimization, indexing, and error handling. ETL and Data Warehousing: Expertise in ETL/ELT pipeline design and orchestration for batch and near real-time processing using SQL Server Agent, SSIS, or Azure Data Factory. Data Modeling: Solid understanding of normalized and dimensional modeling and scalable architecture design. Feature Store Development: Ability to design and maintain reusable feature tables supporting machine learning and operational scoring. Data Validation and Quality Assurance: Skilled in implementing validation rules, reconciliation checks, and QA frameworks to ensure data integrity. Data Governance and Security: Strong knowledge of data governance, privacy, and compliance standards; experience maintaining data lineage documentation. Workflow Orchestration: Experience building restartable, traceable workflows with checkpoint and rollback mechanisms. Programming and Scripting: Proficiency in SQL and beneficial experience in Python or R for automation and data manipulation. Cloud Platforms: Familiarity with Azure or other cloud platforms such as AWS or GCP for data engineering workloads. Version Control and CI/CD: Exposure to Git and CI/CD pipelines for managing data workflow deployment. Visualization and Reporting : Ability to prepare scored or curated data for BI tools such as Power BI. Performance Optimization: Expertise in performance tuning, query profiling, and indexing strategies to optimize large-scale data operations. Collaboration and Communication: Ability to work effectively across technical and business teams, translating complex requirements into practical data solutions.Design, build, and optimize T-SQL stored procedures, functions, and scripts for high-volume data processing and ECM scoring. Develop, deploy, and monitor end-to-end ETL/ELT workflows with checkpoint/rollback, job tracking, and recovery capabilities. Perform data cleansing, preparation, and transformation to support business intelligence and machine learning workflows.Model and maintain data warehouse structures , ensuring proper documentation of data lineage. Prepare and deliver curated, scored datasets for downstream consumption in Power BI dashboards and analytics environments. Develop and maintain audit, telemetry, and job tracking tables to ensure data reliability, restartability, and monitoring visibility. Support and troubleshoot production pipelines, optimizing query performance via indexing, tuning, and profiling tools.Implement and monitor data validation, reconciliation, and QA frameworks across the data lifecycle. Enforce data security, privacy, and compliance controls in line with corporate and regulatory standards. Support the implementation of data governance and lineage documentation, ensuring traceability and adherence to EDM policies.Collaborate with data analysts, data scientists, software engineers, and business stakeholders to translate business problems into scalable data solutions.Contribute to all phases of the SDLC, including requirements, design, development, testing, deployment, and maintenance.A tertiary qualification in Computer Science, Information Systems, Demonstrated delivery of production data solutions—both batch and near real-time—within enterprise environments.Track record of implementing data governance and quality frameworks, ensuring compliance and traceability. Experience in orchestrating complex data pipelines using SQL Server Agent, SSIS, Airflow, or Azure Data Factory.Exposure to Power BI or equivalent visualization tools for reporting and analytics enablement.Key Skills and Competencies: Advanced SQL Server Development: Strong proficiency in T-SQL, stored procedure design, query optimization, indexing, and error handling. ETL and Data Warehousing: Expertise in ETL/ELT pipeline design and orchestration for batch and near real-time processing using SQL Server Agent, SSIS, or Azure Data Factory. Data Modeling: Solid understanding of normalized and dimensional modeling and scalable architecture design. Feature Store Development: Ability to design and maintain reusable feature tables supporting machine learning and operational scoring. Data Validation and Quality Assurance: Skilled in implementing validation rules, reconciliation checks, and QA frameworks to ensure data integrity. Data Governance and Security: Strong knowledge of data governance, privacy, and compliance standards; experience maintaining data lineage documentation. Workflow Orchestration: Experience building restartable, traceable workflows with checkpoint and rollback mechanisms. Programming and Scripting: Proficiency in SQL and beneficial experience in Python or R for automation and data manipulation. Cloud Platforms: Familiarity with Azure or other cloud platforms such as AWS or GCP for data engineering workloads. Version Control and CI/CD: Exposure to Git and CI/CD pipelines for managing data workflow deployment. Visualization and Reporting : Ability to prepare scored or curated data for BI tools such as Power BI. Performance Optimization: Expertise in performance tuning, query profiling, and indexing strategies to optimize large-scale data operations. Collaboration and Communication: Ability to work effectively across technical and business teams, translating complex requirements into practical data solutions.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
January salaries rise ahead of expected Budget reliefIT Industry News. Daily.
Read more »
Data Engineer – Remote RemoteIT Industry News. Daily.
Read more »
Intermediate Data Engineer – Gauteng Pretoria RegionIT Industry News. Daily.
Read more »
Senior Data EngineerIT Industry News. Daily.
Read more »
Intermediate Data EngineerIT Industry News. Daily.
Read more »
Senior Data Engineer – Western Cape Cape TownIT Industry News. Daily.
Read more »