Data Engineer

February 12, 2024

Apply for this job

Job Description

Higher is thrilled to be partnering with an exciting and newly founded AI company, that is developing ground-breaking technology able to model human behavioural data on a massive scale.

The company has just secured a multi-million-pound contract with a well-known UK retailer and is growing. As a result, they are seeking a talented Data Engineer who will work on the data and high-performing systems that underpin cutting-edge forecasting and AI systems.

Our client is building on over 20 years of commercial and academic expertise to implement AI solutions that serve multinational brands and commercial partners while paving the way for social good at previously unseen scales. Built on the success of their existing data science consulting company, the founders established the company as a new business to empower retailers to make smarter decisions. The team are experts in transforming the sparse, incomplete and shifting “digital footprint data” we leave in our wake, into practical insights – producing commercial value for clients and underpinning social good in sectors as diverse as healthcare and the environment. The challenges embedded in behavioural data have led the company to pioneer cutting-edge developments in time series forecasting, behavioural modelling, and explanatory AI.

This Nottingham-based company is currently a team of 7, with scope to grow. They offer a hybrid working model and the team usually gets together around once/twice per week, in a shared workspace in Nottingham.


The Data Engineer will work directly with the leadership team to curate and manage the data and high-performing systems required to underpin ground-breaking forecasting and explanatory AI systems, for major multinational retailers. There will be opportunities to work on additional R&D sprints to pioneer novel technology, systems, and methodologies.


  • You’re an engineer at heart and enjoy the challenge of building reliable, efficient data systems and pipelines
  • You have experience in writing complex queries against relational data stores
  • You have experience in identifying root cause issues of poor data quality
  • You have a good understanding of coding best practices and design patterns and experience with code and data versioning, dependency management, code quality and optimisation, error handling, logging, monitoring, validation and alerting
  • You have a good understanding of design choices for data storage and data processing, with a particular focus on cloud data services (Ideally Databricks)
  • You have experience in programmatically deploying, scheduling and monitoring components in a workflow
  • You have experience in designing data systems that are in accordance with the proper data architecture
  • You have knowledge of securing data within systems


  • 5 years’ experience in a relevant role
  • Experience working on Big Data (e.g. terabyte+ datasets)
  • Highly SQL proficient
  • Experience with Python
  • Experience using Cloud Computing solutions
  • Understanding and experience of QA process
  • Experience in the design, support and management of data dictionaries
  • Previous use of Github/CVS
  • Diligent, methodical and adaptable


  • Experience using Microsoft Azure/SparkSQL/Databricks
  • Experience working with systems focused on machine learning
  • Comfortable working both independently and as part of a team
  • Passionate about implementing and extending systems to enable state-of-the-art AI to augment human intellect


  • Salary range of £65-85k
  • 30 days holiday plus bank holidays
  • A “blanket day” – one day a year where you can stay home no questions asked
  • Innovation days – one day per month for blue sky research – working on whatever makes you curious
  • The opportunity to shape the future of AI development
  • Regular company social get-togethers
  • Company pension scheme
  • The opportunity to participate in Arcaidium’s Share Option Scheme – to share in Arcaidium’s success
  • Annual salary reviews, with the opportunity for significant professional development as Arcaidium grows