Sr Lead Software Engineer
Aumni
This job is no longer accepting applications
See open jobs at Aumni.See open jobs similar to "Sr Lead Software Engineer" NEXT Frontier Capital.Job Description
We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.
As a Sr. Lead Software Engineer in Assurance Foundations Data Team within Cyber Security and Technology Controls, you will partner with established product, architect and security members of the team and serve as a mentor to newly hired and more junior team members to deliver outcomes for Risk Assurance and strategic stakeholder across the control ecosystem. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job responsibilities
- Design, develop, and maintain data ingestion pipelines to collect, process, and transport large volumes of data between various sources, sinks and stores into our data ecosystem
- Collaborate with data engineers, data scientists, and other stakeholders to normalize, correlate and integrate data from diverse sources, ensuring data quality and consistency.
- Identify and resolve bottlenecks in data ingestion processes, optimizing performance and throughput.
- Implement robust monitoring and alerting systems to proactively identify and address data ingestion issues. Troubleshoot and resolve data ingestion problems promptly.
- Ensure data security and compliance by implementing appropriate access controls, encryption, and data protection measures within the ingestion pipelines.
- Maintain comprehensive documentation of data ingestion processes, configurations, and best practices.
- Service Development: Utilize your expertise in a development language (e.g Java, Python) to provide management services for data ingestion solutions, ensuring code quality and performance optimization.
Required qualifications, capabilities, and skills
- Formal training or certification on data engineering concepts and 5+ years applied experience.
- Experience deploying, using and monitoring AWS data services including Glue, Athena or Neptune
- Experience with Graph Databases, including Cypher and Gremlin, or Relational Databases including DML, DDL, and PL/SQL.
- Experience managing deployment, upgrade, redeployment and teardown of Cloud services using Terraform
- Experience integrating data platforms through Continuous Integration & Deployment (CI/CD) - ideally, using git/Bitbucket and Jenkins or spinnaker
- Experience in building data pipelines using Spark, Glue or similar. Strong understanding of data integration concepts, ETL processes, and data modeling.
- Proficient in working with various data formats, including JSON, XML, and CSV.
- Demonstrated execution of full delivery (designing, developing, coding, testing, debugging and documenting) of secure data systems to satisfy business requirements.
Preferred qualifications, capabilities, and skills
- Experience working in an Agile Development environment and is able to contribute/facilitate Agile ceremonies.
- Proficient in full stack development having experience creating REST based services on Java/Spring/Spring Boot.
- Test driven development using modern source control and continuous integration
- Experience with API and GraphQL interfaces. Experience with data ingestion, transports and message busses (e.g EKS, Apache Kafka, Apache Nifi, Pentaho, Apache Hop, or similar)
- Good understanding of Data Warehousing and Data Modelling concepts on AWS Redshift .Knowledge of distributed computing and big data technologies (e.g., Hadoop, Spark)
- Strong interpersonal and communication skills to work collaboratively in and across teams to achieve common goals
This job is no longer accepting applications
See open jobs at Aumni.See open jobs similar to "Sr Lead Software Engineer" NEXT Frontier Capital.