Senior Data Engineer

Responsibilities and Duties

1. 1Design and implement data pipelines to ingest data from multiple source systems (CRM, CBS, CLM, LOS) using REST APIs or database connections
2. 2Build and maintain Bronze/Silver/Gold layer transformations ensuring data quality, consistency, and performance
3. 3Implement data quality checks and cross-system reconciliation logic (e.g., validating CBS transaction records against ledger balances)
4. 4Develop and optimize SQL queries and transformations using Databricks SQL/notebooks or Delta Live Tables
5. 5Design and implement data models for analytics and reporting use cases, with a primary focus on CRM data integration, alongside regulatory reporting and the outbound Conseek risk-export mart
6. 6Build REST APIs or data serving layers for downstream consumers and participate in architecture decisions for data platform components
7. 7Write unit tests, integration tests, and data quality tests for pipelines; monitor performance, troubleshoot failures, and implement improvements
8. 8Implement infrastructure as code using Terraform, set up CI/CD pipelines, mentor mid-level engineers, conduct code reviews, and lead features and initiatives within the data platform

応募資格（必須）

• 5+ years of experience in data engineering or analytics engineering
• Strong proficiency in SQL and Python
• Hands-on experience building data pipelines using modern tools (Databricks, Spark, or similar)
• Experience with cloud data platforms (AWS, Azure, GCP) and storage systems (S3, ADLS, GCS)
• Strong understanding of data modeling techniques including dimensional modeling, data vault, or event-driven architectures
• Proven ability to debug and optimize slow queries and data processing jobs
• Experience with version control (Git) and CI/CD pipelines
• Understanding of data governance concepts: access control, audit logging, data lineage
• Strong problem-solving skills and ability to work independently
• Experience mentoring junior or mid-level engineers
• Excellent communication skills for collaborating with cross-functional teams
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience
• Language ability: Japanese at Business level required

歓迎する経験

• English proficiency (Business level or above, e.g. TOEIC 700+) is a plus for cross-team documentation and collaboration
• Experience with data quality validation and testing frameworks
• Experience in financial services, fintech, or other regulated industries
• Knowledge of banking domain concepts: core banking systems, payment processing, regulatory reporting, AML/transaction monitoring
• Experience implementing data platforms that comply with regulatory requirements (FI
• SC Security Guidelines, FSA/BOJ reporting, GDPR, APPI)
• Hands-on experience with Databricks platform (AutoLoader, Unity Catalog, Delta Live Tables, Databricks SQL, Databricks Workflows)
• Experience implementing cross-system reconciliation for financial data
• Experience with performance tuning on Databricks: partitioning strategies, Z-ordering, query optimization, cost management
• Experience building REST APIs with Python (FastAPI, Flask, or similar) for data serving
• Knowledge of streaming data ingestion patterns and external tools (Kafka, Kinesis)
• Experience with Terraform
• Contributions to open-source data engineering projects
• Experience with Databricks SQL Dashboards or BI tools (Tableau, Looker, PowerBI)
• Experience leading technical initiatives from design through implementation
• Track record of improving data platform performance or reducing costs (provide specific metrics)
• Experience in AI development and/or experience in using AI tools to improve development processes.

待遇・福利厚生

• インフルエンザ予防接種
• ご近所手当
• 家賃給与控除制度

求めるスキル・経験