Data Engineer
Posted
Pay Rate: $77.00 - $82.00 Negotiable based upon years of experienceWork Mode: Hybrid
Global Financial Firm located in IRVING, TX has an immediate contract opportunity for an experienced Data Engineer
Responsibilities:- Partner with project managers, business stakeholders, and senior managers to translate complex business requirements into technical solutions.
- Drive cross-functional collaboration to ensure seamless integration of functions.
- Identify and implement necessary system enhancements for new products and process improvements.
- Lead the resolution of high-impact problems through the evaluation of business processes and system architectures.
- Serve as a subject matter expert in application programming and ensure adherence to architectural blueprints.
- Provide mentorship and technical guidance to mid-level developers and analysts.
- Design, develop, and maintain data pipelines using PySpark.
- Collaborate with data scientists and analysts to understand data requirements.
- Optimize Spark jobs for performance and efficiency.
- Implement data quality checks and ensure data integrity.
- 6-10 years of experience in systems analysis and programming of software applications.
- Strong proficiency in Java application technologies, including TDD, Spring framework, and Microservices architecture.
- Extensive hands-on experience with PySpark and Python programming skills.
- Experience with Big Data ecosystems, including Cloudera and/or Data Bricks.
- Proficiency in SQL and experience with relational and non-relational databases.
- Experience with distributed query engines like Starburst (Trino/Presto).
- Proficient in designing and managing workflows using scheduling tools like Apache Airflow.
- Experience in RESTful API development and integration.
- Experience with job scheduling and Linux environments.
- Experience with AI development tools.
- Interest in the financial services industry.
- Experience in real-time data streaming and processing using PySpark Structured Streaming.
- Knowledge of machine learning concepts and MLOps practices.
- Familiarity with data visualization tools like Tableau or Power BI.
