Spark Data Engineer
Start Date: Apr 1, 2019
Swoon is looking for a Spark Data Engineer for our client in Bradenton, FL! Are you ready to be Swoon'd? See below to see if you qualify!
SUMMARY OF POSITION
We are seeking an innovative individual who has a proven track record of building enterprise level platform components to support product development from multiple teams and lines of business. This role is expected to drive innovation through collaboration across our data science teams and business to help push us to the next level. The team is embarking on a journey of building a brand new data lake platform built using cloud native concepts and the latest tech stacks. Some ideal technologies for this individual would be Spark, Scala/Python, Kafka, and Azure (AWS is ok too)
This role will include, but will not be limited to the following responsibilities:
• Build new data sets, and products helping support business initiatives.
• Help grow our data catalog through ingestions of a variety of third party data sources, both internal and external
• Must be able to contribute to self-organizing teams with minimal supervision working within the Agile / Scrum project methodology
• Build production quality ingestion pipelines with automated quality checks to help enable the business to access all of our data sets in one place
• Participate in the continuous evolution of our schema / data model as we find more data sources to pull into the platform
• Support our Data Scientists by helping enhance their modeling jobs to be more scalable when modeling across the entire data set
• Participate in a collaborative, peer review based environment fostering new ideas via cross team guilds / specialty groups
• Maintain comprehensive documentation around our processes / decision making
• Bachelors Degree in Computer science or related field
• 3+ or more years of related information technology experience
• 2 years of experience in Spark,Scala ,Python,Hive and Data Lake.
• Experience in developing ETL Processes
• Strong SQL Skills
• Good understanding of data warehouse design(e.g. dimensional modeling)
• Demonstrated experience with agile scrum methodology
• Strong desire to learn new technologies and keep up with the latest technologies in the big data space
• Must possess well-developed verbal and written communication skills.
• Public cloud experience, preferably Azure or AWS
• Experience using CI / CD tools and project build automation
o Git, Jenkins, Maven, PyPi, etc
• Previous experience at a fortune 500 company
• Experience with enabling Data Science and Self Service product development with clean, reliable data sets.
The Data Engineer does not have direct supervisory responsibilities but they are expected to demonstrate leadership and mentor other team members.
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. The successful candidate will have a high degree of motivation, exceptional attention to detail, and the ability to derive answers from communications with multiple parts of the business. This person will also require the ability to work well within a team atmosphere and has the flexibility to help with a variety of the department goals.
EDUCATION and/or EXPERIENCE
Bachelor's degree (B. S.) from a four-year college or university; or Associate degree in programming from a technical college; and 3-5 years related experience and/or training; or equivalent combination of education and experience.
The candidate must be able to communicate with others in a simple, clear, and understandable manner. Must also have the ability to read and interpret all forms of communication. The ability to write reports for management and professional correspondence, as well as to speak effectively before peers, management, customers, and other associates of the organization is required.
The candidate must effectively plan and execute unusual and difficult work. Takes initiative and is decisive and quick, when appropriate. All while adhering to the team’s standards and patterns and practices. Must have the ability to solve practical and impractical problems and deal with a variety of concrete variables in situations where only limited standardization exists. Ability to interpret a variety of instructions furnished in written, oral, diagram, or schedule form. Deals constructively with conflict and disagreement.
The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. While performing the duties of this job, the employee is regularly required to sit and talk or hear. The employee must regularly lift and/or move up to 10 pounds. Specific vision abilities required by this job include close vision, and ability to adjust focus.
The work environment characteristics described here are representative of those an employee encounters while performing the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
The noise level in the work environment is usually quiet, however positive, work-driven collaboration is encouraged.