Data Engineer III - Python, Pyspark, Databricks
Aumni
This job is no longer accepting applications
See open jobs at Aumni.See open jobs similar to "Data Engineer III - Python, Pyspark, Databricks" NEXT Frontier Capital.Software Engineering, Data Science
bournemouth, uk
Posted on Jul 18, 2024
Job Description
Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team.
As a Data Engineer III at JPMorgan Chase within the Corporate and Investment Bank for Payments Technology, you will be part of our dynamic team, embarking on a multi-year data strategy program within the Cash Account Management & Intraday Liquidity team. This exciting program is designed to transform our data infrastructure and analytics capabilities, empowering us to make strategic decisions and promote business growth. You will have the opportunity to leverage your strong background in Python development, data engineering, and data strategy implementation. You will play a key role in enhancing the risk management, oversight, control, and reporting related to JP Morgan’s intraday liquidity through the Cash Account Platform (CAP).
This firm-wide, strategic initiative provides real-time dashboards for senior executives, visualizing Cash, Credit, and Collateral balances & activity at the Firmwide, Line of Business, and Institutional Client level. You will also contribute to the firm’s cash forecast and funding functions, working closely with the nostro account reference data system. As part of the Bournemouth development team, you will be one of the global Agile teams actively developing a suite of applications, with a focus on Cash Management.
Job responsibilities
- Develop and Maintain Data Pipelines:
- Design, build, and maintain efficient, reusable, and reliable data pipelines. Integrate data from various sources, including databases, APIs, and flat files. Implement ETL processes to support data transformation and loading. Ensure the robustness and reliability of data processing systems. Develop scripts to automate repetitive tasks and improve data processing efficiency. Ensure scripts are well-documented and maintainable
- Data Management & Migration:
- Work with relational and NoSQL databases to store and retrieve data. Optimize database performance and ensure data integrity. Also contribute to the migration of data from RDMS to cloud data source. Design and implement robust data models to support analytical use cases. Work closely with data analysts, data scientists, and other stakeholders to understand data requirements and deliver solutions. Participate in code reviews and provide constructive feedback.
- Implement Data Strategy:
- Contribute to the development and execution of the data strategy. Assist in the design and implementation of data governance and data quality frameworks.
- Performance Tuning:
- Identify bottlenecks and bugs, and devise solutions to address these issues. Optimize the performance of data processing workflows.
- Documentation:
- Maintain comprehensive documentation for all data processes, pipelines, and systems. Ensure that documentation is up-to-date and accessible to relevant stakeholders.
Required qualifications, capabilities, and skills
- Formal training or certification on Python and relevant libraries concepts and proficient applied experience.
- Proficient at Python and relevant libraries (e.g., Pandas, NumPy, SQLAlchemy).
- Expert in Database, PL\SQL, Performance tuning, DB modelling, Erwin, DB query review, database query optimization.
- Experienced development in a data lake area using Databricks, Redshift or Snowflake tools.
- Experience of working on streaming data applications such as Spark Streaming, Kafka, MSK, Kinesis.
- Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and their data services.
- Knowledge of machine learning and data science principles.
- Understanding of data governance and data quality principles..
Preferred qualifications, capabilities, and skills
- Experience working with Tableau,
- Experience with DevOps including Continuous Integration (CI) and Continuous Deployment (CD) tools e.g. Jenkins, Sonar.
- Exposure to scheduling tools like Autosys / Control-M.
- Experience with big data technologies (e.g. Spark, Hadoop).
- Familiarity with data integration tools (e.g., Apache Airflow) and data warehousing solutions (e.g.Databricks, Redshift).
- Work effectively within Agile development framework to ensure timely and efficient project delivery
This job is no longer accepting applications
See open jobs at Aumni.See open jobs similar to "Data Engineer III - Python, Pyspark, Databricks" NEXT Frontier Capital.