Job Description
We are seeking a skilled Data Engineer to develop and implement web scraping solutions using Python/Golang to collect social media data. The successful candidate will be responsible for building comprehensive user profiles based on underlying data and leveraging social media, user profiles, AI, and other technologies to construct intelligent signals.
This role involves working with large datasets, developing data pipelines, and collaborating with cross-functional teams to deliver actionable insights. The ideal candidate will have a strong technical background and a passion for data-driven decision making.
Key Responsibilities
- Develop and maintain web crawlers using Python/Golang to collect social media data
- Design and implement data processing pipelines for user profile construction
- Build intelligent signal systems integrating social media data, user profiles, and AI technologies
- Optimize data collection and processing workflows for efficiency and scalability
- Collaborate with data scientists and analysts to deliver actionable insights
- Ensure data quality and integrity throughout the data lifecycle
Job Requirements
- Bachelor's degree or higher in Computer Science, Engineering, or related field
- Minimum 1 year of relevant work experience
- Proficiency in Python and/or Golang with experience in data processing
- Familiarity with big data processing tools such as Flink, Kafka, or similar technologies
- Strong problem-solving skills and ability to work independently
- Excellent communication and collaboration skills
- Continuous learning mindset and ability to research new technologies
Benefits
- Minimum 4 months annual bonus
- Statutory holidays
- Flexible work options including Singapore office or remote work
- Opportunities for professional growth and development
- Competitive compensation package
- Collaborative and innovative work environment
