Job Description
The
job holder is responsible for designing, developing, and maintaining
the systems that collect, manage, and convert raw data into usable
formats for data analysis. The job holder ensures that data pipelines,
storage solutions, and data processing systems are robust, scalable, and
optimized for high-quality data analysis and decision-making
job holder is responsible for designing, developing, and maintaining
the systems that collect, manage, and convert raw data into usable
formats for data analysis. The job holder ensures that data pipelines,
storage solutions, and data processing systems are robust, scalable, and
optimized for high-quality data analysis and decision-making
Job Responsibilities
-
Build and maintain scalable, efficient data pipelines to collect, process, and store large volumes of data from various sources
-
Implement Extract, Transform, Load (ETL) processes to clean, structure, and transform data for downstream analysis
-
Design
and manage databases (SQL and NoSQL) to store and retrieve data
efficiently and ensure databases are optimized for performance,
security, and scalability -
Utilize
big data tools and technologies (e.g., Hadoop, Apache Spark, Kafka) to
handle massive datasets and ensure smooth processing -
Work
closely with data scientists, data analysts, and business intelligence
teams to understand data requirements and deliver the necessary
infrastructure. -
Integrate
data from various sources, such as APIs, third-party systems, or cloud
platforms, into the organization’s data ecosystem -
Implement and enforce best practices for data quality, consistency, and integrity across all data systems
-
Leverage cloud platforms (e.g., AWS, Azure, Google Cloud) to store and manage data efficiently and cost-effectively
-
Ensure the data is securely stored and accessed by implementing encryption, access control, and other security best practices
-
Continuously monitor, tune, and optimize data systems and processes to enhance performance and minimize downtime
-
Document
data engineering processes, pipelines, and solutions to ensure that all
workflows are understood, reproducible, and maintainable
Job Requirements
-
Master’s degree in computer science, data engineering, software development, information Technology, or a related field.
-
5 years of progressive experience in software development, DevOps, data engineering or a related field.
-
Professional
Certifications in workflow orchestration, cloud platforms, big data
technologies, or database management (e.g., Apache Airflow
Certification, AWS Certified Data Analytics, Google Professional Data
Engineer, or similar) is added advantage
Leave a Reply