Job Description
Lead the development of solutions to overcome complex anti-crawling mechanisms, maintain large-scale distributed crawling systems, and ensure stable data acquisition from core sources. Deeply understand quantitative trading logic, collaborate closely with the team to translate business requirements into efficient technical solutions. Work closely with product, design, and frontend teams to ensure high-quality deliverables.
Key Responsibilities
- Design and maintain robust distributed crawling systems to handle anti-scraping challenges
- Develop technical solutions that align with quantitative trading business requirements
- Collaborate cross-functionally with product managers, designers and frontend developers
- Ensure system stability and data quality for core data sources
- Optimize crawling performance and efficiency through technical innovation
Job Requirements
- Solid Engineering Foundation:
- 3+ years Python development experience with strong CS fundamentals (data structures, algorithms, networking, OS)
- Deep understanding of Python internals (GIL, memory management), proficient with process/thread/coroutine models
- Experienced with FastAPI/Django frameworks, MySQL/Redis, database design and optimization
- Proficient with Docker, familiar with Kubernetes, capable of independent deployment
- Crawling & Reverse Engineering:
- 2+ years large-scale data collection experience with Scrapy/Playwright frameworks
- Strong JS reverse engineering skills to debug and deobfuscate code
- Familiar with encryption algorithms (RSA/AES/DES) in anti-scraping
- Expertise in anti-scraping mechanisms (CAPTCHAs, IP blocking, request signing, behavior detection)
- Experience with data cleaning and storage solutions for structured/unstructured data
Preferred Qualifications
- Practical trading experience with understanding of market trends
- Experience applying ML/DL in candlestick pattern recognition or algorithmic trading
- Familiarity with digital currency, futures or securities markets
- Strong product design thinking
- AI/ML implementation experience
- Golang proficiency
Benefits
Please contact HR directly via Telegram for benefits discussion.
