Job Overview
As a Senior Software Engineer at the Data Platform engineering team, you will play a pivotal role in designing, developing, and optimizing our data platforms and building scalable and robust data systems that handle health data. Your expertise will be crucial in shaping our data strategy and ensuring the seamless integration and performance of our data infrastructure.
Responsibilities
Participate in design and architecture of innovative data systems from scratch to production to support Briya’s query and data platform (scalable, reliable, and high-performance data processing)
Design, develop and maintain ETL workflows for clinical data sources, EMRs etc. for data cleaning, transformation, and aggregation and for facilitating data query
Develop backend services which perform data transformations to standardise data and prepare it for query and research
Troubleshoot and resolve issues related to data integration, storage, and retrieval.
Optimize database performance and query execution in large-scale environments.
Analyze raw data and maintain datasets specific to business requirements
Collaborate with external and internal stakeholders for assistance in data-based issues (data quality, data accessibility etc.)
Stay up-to-date with best practices to continuously improve our technology stack and development processes of backend engineering and big data processing.
Requirements and Skills
7+ years of proven back-end development:
Experience and proven experience working in a micro-services environment and cloud platforms
Strong proficiency in programming languages such as Go, Java, Python, or Scala.
Strong experience with Linux OS and runtime
5+ years experience as a data engineer or a similar role of working with data:
Experience with big data technologies and data platforms (e.g. Elasticsearch, Snowflake, Spark, Kafka, Athena, Trino)
Technical expertise with data models, data analytics and processing
Solid understanding of SQL, data modeling, ETL processes, and data warehousing.
Excellent problem-solving skills and the ability to think critically and creatively
An enthusiastic self-learner who can confidently embrace new technologies, conduct research to discover innovative solutions, and successfully deliver results
Strong communication skills and the ability to collaborate effectively with cross-functional teams
Ability to adapt quickly and effectively manage context switches
Preferred Skills
Experience with containerization technologies like Docker and Kubernetes.
Knowledge of machine learning and data analytics or building analytical systems.
Familiarity with DevOps practices and tools (e.g. Terraform, Helm)
Good understanding of relational DBs internals
Strong understanding of Functional programming