Technical - you need to be strong in fundamentals. Syntax is not important Problem solving/ solutioning - approach is important. Not syntax Managerial - You should be able to articulate your projects well. Project goals, scope, value ads are important - not technologies Behavioral - Be transparent
In First Round Mostly questions related to basics of spark and Hadoop questions. In Second Round Question related to optimization and fine tuning of spark jobs, Skew Joins
1.Questions on Spark RDD & DataFrames and data loading from different sources.
Design pipeline that is relevant to the company
Context Given a night with X ride requests and Y available drivers in a fictional city, you have to develop a batch processing application that would aggregate and expose data coming from the matching engine. Every Z seconds, the matching engine tries to match every pair of request and driver that are available in the city. Some are matched, some are not. The results of each matching tick are stored in a set of files. Given this data we want to be able to get a overview of the marketplace health and multiple applications could follow such as heat maps. The metrics that we want to use for heat mapping are driver match rate and request match rate. Exercice Develop a microservice that will: Aggregate matching data by fetching new matching data and exploiting this data in order to aggregate driver match rates and request match rates by geo-spatial units of your choice. Expose an endpoint which return the adjusted values of request and driver match rate, following this formula.. Bonus Plot the driver and request match rate for one night
Topics: 1. SQL- Window functions 2. Bigdata Ecosystem 3. Design questions (ETL design)
Tell me the time when you had worked with SQL
Tell me about a time where you had to adapt to an unexpected change
Data architecture, work experience, behavioral questions
I had 4 rounds of interview.
Viewing 1511 - 1520 interview questions