Senior Big Data Engineer – Spark/Scala
We are currently on the lookout for multiple Big Data Engineers at both Lead and Senior levels as our clients look to grow the team here in Ireland.
The client who provides a cloud-based big data platform amongst other products is looking for Spark and Scala experience but is open to someone coming from a Python background.
- Actively participate in team technical discussions in all things data
- Identify and address issues with data sets from multiple vendors
- Identify and address code and data quality issues
- Actively participate in code reviews and grooming sessions
- Actively participate in technology architecture discussions for product development
- Translate business requirements into strategy
- Advocate for software best practices within your team as well as across engineering
- Be ultra-responsive and capable of making instant decisions, always kicking the ball forward
- Work on unique and interesting data challenges around architecting, building and managing pipelines that securely process hundreds of terabytes of data
- Work closely with analysts and statisticians to ensure the validity of our processes
- Bachelor's degree in Computer Science or a related field (or 4 additional years of relevant work experience)
- A strong understanding of data structures, algorithms, and effective software design
- Significant development experience with a major modern language (e.g. Scala, Python)
- Skillful user of Apache Spark
- Experience working with structured and unstructured data at scale and comfort with a variety of different stores (key-value, document, columnar, etc.) as well as traditional RDBMSs
- Experience with the full development lifecycle, from ideation to running software in production
- Excellent verbal and written communication skills; must work well in an agile, collaborative team environment
- Experience with AWS products (Redshift, EMR, S3, IAM, RDS, etc)
- Experience with or interest in Databricks and any other tools that enable data processing at scale
- Experience with automation tools such as Apache Airflow
Get in touch and I can provide you with more info. Email: email@example.com or call 0862247895.