The position is based out of Pune, India and will report to senior big data engineer. There is an opportunity to use modern big data technologies. Candidate will work independently with business owners, global teams & stake holders. Applies skills and knowledge of the tools to develop creative solutions to meet client and business needs. Person would be involved in requirement gathering, analysis, design, Coding, unit testing, review, integration testing, QA testing cycle, release-deployment cycle for services under CCB big data program.
So, will learn all development processes followed in software sprint cycle and QA and release cycles.
**Responsibilities:**
**Education:**
+ We are looking for a Spark data engineer who knows how to fully exploit the potential of our Spark cluster.
+ You will clean, transform, and analyze vast amounts of raw data from various systems using Spark\Pyspark to provide ready-to-use data to our feature developers and data scientist.
+ This involves both ad-hoc requests as well as data pipelines (batch and real time streaming) that are embedded in our production environment.
+ Taking ownership to deliver projects by full development lifecycle of project.
+ Create Scala/Spark jobs for data transformation and aggregation.
+ Ability to build analytical model using python\Spark.
+ Good to have Kafka or spark streaming knowledge.
+ There will be interaction with internal business partner/ data scientist.
+ Develop and manage a communication program to ensure that the partner organizations are kept apprised of progress of conducting periodic reviews with work stream Business Leads and Key Stakeholders
+ Contribute to formulation of strategies for applications development and other functional areas
+ Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
+ Provide evaluative judgment based on analysis of factual data in complicated and unique situations
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards.
+ **Qualifications:**
+ 1-3 years of relevant experience in the Financial Service industry
+ Basic knowledge of industry practices and standards
+ Consistently demonstrates clear and concise written and verbal communication
+ Scala (with a focus on the functional programming paradigm) or Pyspark (with in depth python knowledge)
+ Apache Spark 2.x
+ Apache Spark RDD API
+ Apache Spark SQL DataFrame API
+ Apache Spark MLlib API
+ Apache Spark GraphX API
+ Apache Spark Streaming API
+ Spark query tuning and performance optimization
+ SQL database integration **{{ Microsoft, Oracle, Postgres, and/or MySQL }}**
+ Experience working with **{{ HDFS, S3 }}**
+ Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
+ Good to have machine learning knowledge (Regression, clustering, classification)
+ Hands-on experience in Unix shell scripting.
+ Bachelors degree/University degree or equivalent experience
+ Masters degree preferred
-------------------------------------------------
**Job Family Group:**
Technology
-------------------------------------------------
**Job Family:**
Applications Development
------------------------------------------------------
**Time Type:**
Full time
------------------------------------------------------
Citi is an equal opportunity and affirmative action employer.
Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
Citigroup Inc. and its subsidiaries ("Citi) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review **Accessibility at Citi (https://www.citigroup.com/citi/accessibility/application-accessibility.htm)** .
View the "EEO is the Law (https://www.dol.gov/sites/dolgov/files/ofccp/regs/compliance/posters/pdf/eeopost.pdf) " poster. View the EEO is the Law Supplement (https://www.dol.gov/sites/dolgov/files/ofccp/regs/compliance/posters/pdf/OFCCP\_EEO\_Supplement\_Final\_JRF\_QA\_508c.pdf) .
View the EEO Policy Statement (http://citi.com/citi/diversity/assets/pdf/eeo\_aa\_policy.pdf) .
View the Pay Transparency Posting (https://www.dol.gov/sites/dolgov/files/ofccp/pdf/pay-transp\_%20English\_formattedESQA508c.pdf)
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
