Description
Professional Summary Talented and accomplished Software Engineer with 8 years of experience in developing applications using Big Data, AWS, Java, SQL and Spark. Hadoop Ecosystem: * 8 years of overall IT experience in Java and Big Data. * 5 years of experience with Big Data tools like Map Reduce, YARN, HDFS, Hbase, Impala, Hive, Pig, Oozie, AWS, Zeppelin, Apache Spark for ingestion, storage, querying, processing and analysis of data. * Performance tuning in Hive& Impala using multiple methods limited to dynamic partitioning, bucketing, indexing, files compressions. * Hands on experience with data ingestion tools Kafka, Flume and workflow management tools Oozie and Zena. * Hands on experience handling different file formats like JSON, AVRO, ORC, Parquet and compression techniques like snappy, zlib and lzo. * Experience on analyzing data in NOSQL databases like Hbase and Cassandra and its Integration with Hadoop cluster. * Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture. Spark Core, SQL and Streaming: * Developed spark applications for data transformations and loading into HDFS using RDD, DataFrames and Datasets. * Extensive knowledge on performance tuning of Spark applications and converting Hive/SQL queries into Spark transformations. AWS Stack: * Hands-on experience with AWS (Amazon Web Services), using Elastic MapReduce (EMR), creating and storing data in S3 buckets and creating Elastic Load Balancers (ELB) for Hadoop front end Web UI's. * Extensive knowledge on creating Hadoop cluster on multiple EC2 instances in AWS and configuring them through ambari and using IAM (Identity and Access Management) for creating groups, users and assigning permissions. Java & Other: * Extensive programming experience in Java Core concepts like OOPS, Multithreading, Collections and IO. * Experience using Jira for ticketing issues and Jenkins for continuous integration. * Extensive experience with UNIX commands, shell scripting and setting up CRON jobs. * Experience in software configuration management using Git. * Good experience in using Relational databases Oracle & MySQL. * Able to assess business rules, collaborate with stakeholders and perform source-to-target data mapping, design. * Successfully working in fast-paced environment, both independently and in collaborative team environments.
Accomplishments
Highlights:
Keywords
