IT資料工程師(Data Engineer)
面議（經常性薪資達4萬元或以上） 40000元
新北市汐止區
1~2年工作經驗
【Job Description】
We are seeking a highly skilled and motivated IT Data Engineer to join our team in Taiwan. This role involves designing, building, and maintaining scalable data pipelines, ensuring data reliability, and enabling insights that drive critical business decisions. The ideal candidate will have a strong background in distributed systems, database management, and data analytics tools, combined with a passion for innovation and continuous improvement.
【Responsibilities】
1. Data Pipeline Development
- Design, develop, and optimize scalable data pipelines using Hadoop, Spark, and Kafka.
- Ensure real-time and batch data processing pipelines meet business requirements.
2. Database Management
- Write complex SQL queries and manage RDBMS platforms such as Oracle, MSSQL, and PostgreSQL.
- Implement and maintain data models and warehouses for analytics.
3. Workflow Automation
- Develop and maintain automated workflows using tools like Apache Airflow.
4. Programming and Development
- Build efficient and maintainable solutions using Python or Scala.
- Collaborate with cross-functional teams to integrate data solutions.
5. Containerization and Deployment
- Leverage Docker and Kubernetes to deploy and manage applications in a containerized environment.
6. Data Visualization and Analytics
- Create interactive dashboards and reports using Qlik, Tableau, or PowerBI.
7. System Administration and Scripting
- Perform Linux system administration and scripting to support data infrastructure.
【Skills】
1. Essential:
- Proven experience with distributed systems like Hadoop, Spark, and Kafka.
- Strong SQL skills and hands-on experience with Oracle, MSSQL, and PostgreSQL.
- Proficiency in Python or Scala for data engineering tasks.
- Experience with workflow schedulers like Apache Airflow.
- Familiarity with Docker, Kubernetes, and containerization technologies.
- Solid understanding of data modeling and warehousing principles.
- Knowledge of Linux administration and shell scripting.
- Expertise in using BI tools such as Qlik, Tableau, or PowerBI.
2. Desirable:
- Experience implementing lakehouse architectures and managing Iceberg tables on S3 object storage (a plus).
- Ability to develop and deploy REST APIs and web services.
- Familiarity with AI/ML pipelines and associated tools.
3. Preferred Skills:
- Strong analytical and problem-solving skills.
- Effective communication and teamwork abilities.
- Ability to work in a fast-paced, dynamic environment with attention to detail.
展開
