Data Engineering questions: Explain data solution Toughest tasks you had a a a data engineer
Senior Data Engineer Interview Questions
2,556 senior data engineer interview questions shared by candidates
SQL Query Writing: **Table A** Columns: col1 col2 **Table B** Columns: col3 col4 There is no common column to implement the JOIN condition and the Required Output is: col1, col2, col3, col4
SQL Query Writing: **Table EMP** Input Columns: EMPID, DEPID, SAL, DOJ Output Columns: EMPID, DEPID, SAL, DOJ, NEXT_SAL NEXT_SAL can be derived as the salary of immediate next employee in the given records.
Which AWS services you have used (Hands-On)? Tell me one use case and one limitation of AWS Lambda function?
1. Explain the difference between SCD Type1, SCD Type2, SCD Type3
It was mostly decided based on client interview
The HR interview was just a relaxed chat based around values and general compatibility for the role. They ensured that you have done your reading around the company and role. The tech challenge was python based, using a provided API to gather/filter/resubmit data. You had to upload the solution along with appropriate tests to Git. The engineer interview didn’t cover the tech challenge solution at all (surprisingly). It was a general discussion on past experience and projects. Later it switched gears to some more general questions like ‘what is etl?’ or ‘tell us about data streaming products?’ or ‘what problems have you had using semi-structured or unstructured data?’. AWS experience and services used was also covered. The whiteboard session I found the hardest part and was again scenario based where you had to diagnose problems in a process and work with the other engineers to come up with a solution. Later (once the main problem had been covered) more requirements/problems were added to see how you thought on your feed and you were questioned on which products you would use (and why)
Interviewer mostly asked spark and python related questions as the position was mainly from ETL development
1> Technical questions related to job role 2> Current project work flows 3> Some programming and Infra related questions 4> English test including speaking, writing and others
Big query optimization techniques (partitioning and clustering)
Viewing 731 - 740 interview questions