Naeem S

Data Engineer

Datalyticx24 • Temps plein

Aug 2018 - Jun 2022 • 3 yrs 10 mos

• Constructed a data pipeline to process semi-structured data by integrating 400 million raw records from 20+ sources using Elastic Search, Google Cloud Platform, and Python • Integrated data from multiple third-party systems for social media monitoring, customer engagement, and voice analytics • Developed REST APIs with Django and Gunicorn, created Docker images, and deployed them on serverless infrastructure. • Build web scraping tools for data extraction from social media sites using Selenium and BeautifulSoup • Integration of machine learning models with an AI platform • Batch and real-time data processing pipeline for machine learning models to make predictions on raw data • Developed a distributed, scalable, unsupervised crawler with Scrapy, Redis, and MySQL. Used as a news aggregator and collected millions of articles weekly • Build and maintain Elastic Search clusters & indices. Created mapping, aggregations, and dashboards • Developed, refined, and scaled data management and analytics procedures, systems, workflows, and best practices • Created basic dashboards for data analysis using Tableau and PostgreSQL • Led a team consisting of one data engineer and two data scientists, collaborating with various stakeholders