USA_Engineer
Pay Rate Range: $53.57 - 55.14/hr.
GBaMS ReqID: (phone number removed)
AWS Databricks
Job Descriptions:
What you will be responsible forAs a Data Engineer| you will Use your understanding of large scale data processing and analytics to wrangle our unique cybersecurity data and create automation| analyses and tools that point to the most significant business| governance| and risk management impacts.
Participate in the design and buildout of petabyte scale systems for high availability| high throughput| data consistency| security| and end user privacy| defining our next generation of data analytics tooling Build data modeling| automation| and ELT workflows to produce Raw| Rationalized| co-Related| and Reporting data flows for graph| timeseries| structured| and semi-structured cybersecurity data Education Qualifications Minimum Qualifications B.S.| or M.S. in Computer Science or equivalent work experience 5 years of experience building large scale distributed systems and data analytics processes on cloud native| in-memory| and fit-for-purpose hybrid infrastructure.
Experience with cybersecurity data and globally distributed log event processing systems with data mesh and data federation as the architectural core is highly desirable.
Experience in big data technologies like PrestoTrino| Spark Flink| Airflow Prefect| RedPanda Kafka| Iceberg Delta Lake| Snowflake Databricks| MemGraph Neo4J as well as modern security tooling like Splunk| Panther| Datadog| Elastic| Arcsight etc.
Experience designing and building data warehouse| data lake or lake house using batch| streaming| lambda and data mesh solutions and with improving efficiency| scalability| and stability of system resources.
Experience working with data warehouses or Databases like Snowflake| Redshift| Postgres| Cassandra etc
Experience writing and optimizing complex SQL and ETL development and designing and building data warehouse| data lake or lake house solutions.
Experience building Data APIs and integrations using tools like GraphQL| Apache Arrow| gRPC| ProtoBuf| designing large scale stream processing systems with Flink| Kafka| NiFI| and similar technologies.
Experience with distributed systems and distributed data storage and large-scale data warehousing solutions| like BigQuery| Athena| Snowflake| Redshift| Presto| etc.
Experience working with large datasets and best in class data processing technologies for both stream and batch processing| graph and time series data| notebooks and analytic visualization environments.
Strong communication and collaboration skills particularly across teams or with functions like data scientists or business analyst.
Preferred Experience
5 years of experience with Python| Java| or similar languages| with cloud infrastructure (e.g. AWS| GCP| Azure)| and deep experience working with big data processing infrastructures and ELT orchestration
Experience developing distributed batch and real-time feature stores| and developing coordinated batch| streaming and online model execution workflows| building and optimizing large scale data proc
Skills: Category Name Required Importance Experience SkillCategoryTest1_MN Databricks Yes 1 > 7 years
GBaMS ReqID: (phone number removed)
AWS Databricks
Job Descriptions:
What you will be responsible forAs a Data Engineer| you will Use your understanding of large scale data processing and analytics to wrangle our unique cybersecurity data and create automation| analyses and tools that point to the most significant business| governance| and risk management impacts.
Participate in the design and buildout of petabyte scale systems for high availability| high throughput| data consistency| security| and end user privacy| defining our next generation of data analytics tooling Build data modeling| automation| and ELT workflows to produce Raw| Rationalized| co-Related| and Reporting data flows for graph| timeseries| structured| and semi-structured cybersecurity data Education Qualifications Minimum Qualifications B.S.| or M.S. in Computer Science or equivalent work experience 5 years of experience building large scale distributed systems and data analytics processes on cloud native| in-memory| and fit-for-purpose hybrid infrastructure.
Experience with cybersecurity data and globally distributed log event processing systems with data mesh and data federation as the architectural core is highly desirable.
Experience in big data technologies like PrestoTrino| Spark Flink| Airflow Prefect| RedPanda Kafka| Iceberg Delta Lake| Snowflake Databricks| MemGraph Neo4J as well as modern security tooling like Splunk| Panther| Datadog| Elastic| Arcsight etc.
Experience designing and building data warehouse| data lake or lake house using batch| streaming| lambda and data mesh solutions and with improving efficiency| scalability| and stability of system resources.
Experience working with data warehouses or Databases like Snowflake| Redshift| Postgres| Cassandra etc
Experience writing and optimizing complex SQL and ETL development and designing and building data warehouse| data lake or lake house solutions.
Experience building Data APIs and integrations using tools like GraphQL| Apache Arrow| gRPC| ProtoBuf| designing large scale stream processing systems with Flink| Kafka| NiFI| and similar technologies.
Experience with distributed systems and distributed data storage and large-scale data warehousing solutions| like BigQuery| Athena| Snowflake| Redshift| Presto| etc.
Experience working with large datasets and best in class data processing technologies for both stream and batch processing| graph and time series data| notebooks and analytic visualization environments.
Strong communication and collaboration skills particularly across teams or with functions like data scientists or business analyst.
Preferred Experience
5 years of experience with Python| Java| or similar languages| with cloud infrastructure (e.g. AWS| GCP| Azure)| and deep experience working with big data processing infrastructures and ELT orchestration
Experience developing distributed batch and real-time feature stores| and developing coordinated batch| streaming and online model execution workflows| building and optimizing large scale data proc
Skills: Category Name Required Importance Experience SkillCategoryTest1_MN Databricks Yes 1 > 7 years
