グローバルでサービスを展開する大手外資系ITサービス企業でのTechnology Lead, Big Dataの求人

[Responsibilities]
Lead an engagement in the space of Big Data with 10-15 members.
Interact directly with end-customer as well as lead and manage technical team.
Report into Enterprise Architect from technical point-of-view and Delivery Manager from delivery perspective.
Experience with building Big Data platforms and areas of data governance, data security, data quality and related tools.
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.
Work with QA Test analyst to ensure test coverage (Including Integration & Regression testing)
Develop new program logic and/or assembles standard logic modules to create new applications

[Qualifications & Experience]
10 Years of overall technology experience
At least 5+ years of hands on experience with Spark, Scala
Experience with relational databases (Preferably Oracle)
Programming SPARK in Scala & proficiency in SQL to write complex SQL queries
Strong data analysis and troubleshooting skills
Domain knowledge of Capital Market is plus
Knowledge of shell scripts and other languages including Python, R, Java is plus
Good understanding of AWS & Open Source big data technologies Airflow, Terraform, AWS Glue, AWS Lambda, Spark SQL
Ability to implement both batch and streaming data pipelines in the AWS and change data capture (CDC) experience
Experience with Databricks preferable
Good knowledge of Linux OS and Shell scripting
Experience working on complex distributed information systems
Experience with version control systems, preferable SVN, git.