大手証券会社でのSenior Big Data Engineerの求人

Key objectives critical to success
We are looking for a talented and experienced professional to join our team as a Big Data Engineer with specific focus on data engineering, automation and innovation. In this role, you will part of a team leading the design, development, and implementation of robust and scalable application which is used for the liquidity reporting. This role is technical and hands-on and requires a deep understanding in big data, Hadoop, mapr, minio, scala / spark, DevOps practices, and software development lifecycle (SDLC). It requires the development and implementation of processes, policies, standards, and solutions in collaboration with the Global treasury users and key stakeholders (e.g., Technology, business, EIS, compliance). You will play a key role in shaping our regulatory platform strategy and ensuring the resilience and effectiveness of liquidity reporting.

2) Responsibilities:
Key Responsibilities
Architect, develop, and enhance large-scale data processing platform using Scala and Apache Spark on Hadoop clusters.
Design and implement robust ETL workflows to ingest, cleanse, and transform financial data from diverse sources to support regulatory requirements such as Basel III and liquidity reporting (LCR/NSFR/PRA).
Manage and optimize object storage solutions using MinIO for scalable data lake architectures.
Integrate and federate data sources using Dremio and Denodo to enable seamless data access for regulatory, risk, and business users.
Collaborate with business users to deliver reliable and high-quality data for regulatory reporting, including liquidity and risk metrics.
Implement best practices for data quality, data governance, security, and regulatory compliance (Basel III, LCR, NSFR, PRA, SOX, FINRA).
Tune cluster resources to ensure system performance, data integrity, and cost-effectiveness.
Provide technical leadership, perform code reviews, and mentor junior engineers.
Document data architectures, workflows, and maintain up-to-date technical documentation, particularly for regulatory data flows.
Stay current with emerging data technologies, banking regulations, and trends in financial risk and compliance.

Mandatory:
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
6+ years of experience in big data/data engineering roles, ideally within financial services.
Expert-level proficiency in Scala programming and advanced use of Apache Spark for large-scale data processing.
Deep knowledge and hands-on experience with Hadoop Distributed File System (HDFS) and ecosystem tools.
Experience administering and utilizing MinIO for scalable object storage.
Solid experience with Dremio and/or Denodo for data virtualization and integration.
Strong SQL skills and experience supporting BI/reporting solutions, particularly Tableau.
Solid understanding of security and governance in financial data management.
Direct experience working with or supporting Basel III (including LCR, NSFR, PRA) or other regulatory liquidity reporting.
Proven ability to optimize data pipelines/workflows for performance and reliability.
Experience working with complex, sensitive, and/or regulated financial data.
Excellent communication and collaboration skills.
Excellent analytical, problem-solving, and project management skills.
Strong communication and interpersonal skills to collaborate with diverse teams and stakeholders.

Preferred Qualifications:
Familiarity with data modeling and data warehouse concepts.
Experience integrating or supporting regulatory frameworks in a financial context.
Exposure to cloud-based data platforms (AWS, Azure, or GCP) is a plus.
In-depth knowledge of regulatory requirements and reporting relevant to finance (Basel III, LCR, NSFR, GDPR, SOX, FINRA, etc.).
Experience with Agile/Scrum methodologies.