This job board retrieves part of its jobs from: Emploi Étudiant | Toronto Jobs | Work From Home

Tech jobs every day in Canada!

To post a job, login or create an account |  Post a Job

   Canadian Tech jobs   

Website updated every day with fresh jobs in the IT industry

Slide 1
Slide 2
Slide 3
previous arrow
next arrow

Senior Data Engineer

Procom Consultants Group Ltd

This is a Contract position in Toronto, ON posted January 2, 2022.

Senior Data Engineer On behalf of our client, Procom Consultants Group is currently looking for a Senior Data Engineer.

Senior Data Engineer Job Details Analyze the source data, prepare data model & mapping, and develop Scala/Spark programs and related components in the work areas of Information Management (“IM”) relating to claims, insurance and finance data.

Use data pipelines to extract data from data sources which are in various formats (viz
– flat files, XML, relational tables, Oracle logs), and use tools (such as StreamSets, Scala and Spark programs) to transform and store the data in big data platform (Data Lake) after data validation.

Deliverables: 1.

Develop Spark and Scala pipelines for data ingestion and Data transformation as per the mapping document 2.

Data Analysis as per requirements and develop data models/mapping 3.

Data validations and automated data quality pipelines 4.

Relevant documentation for sustainment Senior Data Engineer Mandatory Skills This position would typically be met through post secondary education related to computer systems programming and a few years of related experience.

Applied knowledge in Big Data platforms, ideally with exposure to Hadoop ecosystem (HDFS, Pig, Hive, SPARK, Big SQL, NoSQL, YARN); Solid experience in Scala Experience in designing efficient and robust data pipelines using Spark; Excellent skill in SQL development; Knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases; Hands-on experience with structured and unstructured databases; Experience with enterprise systems such as Guidewire Claim Center, Guidewire Policy Center would be an asset; Senior Data Engineer Start Date January 31, 2022 Senior Data Engineer Assignment Duration July 29, 2022 (depending on start date) Senior Data Engineer Working Location Fully remote but PST hours