Job Description
The Data Engineer will be responsible for building and maintaining the company's comprehensive data infrastructure, covering both batch and real-time data processing pipelines. This role requires expertise in data architecture, ETL/ELT processes, and data warehouse design to support business intelligence and decision-making.
Key Responsibilities
- Design and implement end-to-end data systems including both offline batch processing and real-time streaming pipelines
- Develop and maintain ETL/ELT processes for data collection, synchronization, cleaning and transformation to ensure accurate and efficient data flow
- Implement real-time data processing using Flink for collection, cleaning, aggregation and metric calculation
- Architect data warehouse layers and implement data modeling, dimension design and unified metric standards
- Develop data interfaces, reports and basic data services to support business departments (operations, sales, product)
- Monitor data quality, troubleshoot issues and optimize processes to ensure accuracy, completeness and timeliness
- Respond to business data needs quickly and optimize existing data workflows and SQL/job scripts
Job Requirements
- Bachelor's degree or above in Computer Science, Mathematics or related fields
- 5+ years of experience in data development and data warehousing
- Proficient in SQL with ability to write complex queries and stored procedures
- Experience with databases/warehouses like MySQL, Hive, Paimon, HBase
- Programming skills in Java/Python for ETL scripts and data processing jobs
- Familiarity with Spark, Flink, Kafka for data processing and optimization
- Understanding of data warehouse modeling principles and design
- Strong problem-solving skills for data anomalies and performance issues
- Experience with Alibaba Cloud data platforms (MaxCompute, DataWorks etc.) is a plus
- Background in finance, trading or payment domains preferred
Benefits
Please contact our HR team via Telegram for detailed benefits information.
