If you are an AWS Data Engineer with 2-4 years of experience, open to a 100% fully remote role while working for a fast-growing FinTech / Payments software company, and would accept a full-time salary in the $95,000-125,000 range - then look no further!
We are seeking a mid level AWS Cloud Engineer to join our team, focusing on the design, implementation, and maintenance of a robust AWS Data Lake using Redshift , Glue , and related AWS services. This role will also involve critical tasks like validating the data entering the lake, optimizing data pipelines, and ensuring the security and compliance of the entire cloud environment. The ideal candidate will possess a strong technical background in cloud infrastructure, AWS engineering, data quality, and security best practices.
In this role, you’ll collaborate with cross-functional teams to implement scalable solutions that support data-driven business insights and ensure the integrity, security, and availability of enterprise data assets.
Key Responsibilities:
Cloud Engineering & Data Lake Management:
Data Lake Architecture & Management:
Design, build, and maintain scalable AWS Data Lake architectures leveraging services such as Amazon Redshift , AWS Glue , Amazon S3 , and Amazon Athena .
Develop and optimize ETL pipelines to ingest, transform, and store large volumes of structured and unstructured data into the data lake.
Design and implement strategies for efficient data partitioning, indexing, and organization within Redshift and S3.
Integrate real-time and batch data ingestion methods to ensure a comprehensive and up-to-date data lake environment.
Data Validation & Quality Assurance:
Implement data validation processes to ensure that incoming data meets business requirements for accuracy, consistency, and completeness.
Conduct data profiling, identify potential quality issues, and implement fixes or automated remediation procedures to address them.
Define and implement data governance standards, including data lineage and metadata management, to track and manage data flows and transformations.
AWS Infrastructure Engineering:
Cloud Infrastructure Management:
Manage and optimize the AWS infrastructure , including the provisioning of EC2 instances, Lambda functions, and VPC networks to support data lake workloads.
Automate infrastructure management tasks using AWS CloudFormation , Terraform , or other infrastructure-as-code ( IaC ) tools.
Ensure the scalability, availability, and resilience of AWS resources supporting the data lake, including designing for high availability and fault tolerance.
Configure and optimize AWS services such as Amazon RDS , AWS Glue , AWS Lambda , and Amazon S3 for efficient data storage and processing.
Monitoring & Performance Tuning:
Implement and maintain robust monitoring and logging solutions using AWS CloudWatch , CloudTrail , and third-party monitoring tools to ensure performance, security, and compliance.
Analyze system performance and troubleshoot issues related to data processing, storage, and querying (e.g., slow Redshift queries, Glue job failures).
Continuously optimize AWS costs and performance, including cost-saving measures for storage and compute resources in Redshift and S3.
Security & Compliance in AWS:
Cloud Security & Compliance:
Design and implement security best practices for AWS services, ensuring data protection, confidentiality, and compliance with industry standards (e.g., GDPR, PCI-DSS ).
Configure and enforce access controls using IAM roles, AWS SSO , and AWS KMS for data encryption at rest and in transit.
Implement security monitoring tools such as AWS GuardDuty , AWS Security Hub , and Amazon Macie to detect and respond to potential threats.
Ensure proper data encryption using AWS-native services like AWS KMS , and enforce strong authentication methods such as multi-factor authentication (MFA).
Review and maintain AWS security configurations, such as Security Groups , VPC Security , and NACLs , to ensure compliance with security policies and best practices.
Audit & Governance:
Develop and implement security and compliance auditing practices, utilizing AWS Config , CloudTrail , and other audit logging tools to maintain a full audit trail of activity within the cloud environment.
Enforce strict data access controls and data sharing policies to ensure only authorized users have access to sensitive data.
Maintain ongoing compliance with AWS best practices for security, disaster recovery, and business continuity planning.
Required Qualifications:
Proven experience with designing and managing ETL pipelines , particularly with AWS Glue and Redshift.
Strong background in data validation , data quality assurance, and ensuring data consistency across large data environments.
Experience with cloud infrastructure management , including automation with CloudFormation or Terraform .
Practical experience in securing cloud environments, configuring IAM roles, using AWS KMS for encryption, and applying best security practices .
Expertise in SQL and Redshift query optimization .
Experience with AWS monitoring and logging tools such as CloudWatch , CloudTrail , and AWS Config .
Familiarity with data governance and metadata management solutions.
Knowledge of infrastructure as code ( IaC ) principles and tools like CloudFormation , Terraform , or Ansible .
Ability to script and automate tasks using Python or Shell scripting .
Knowledge of cloud security frameworks and regulatory requirements (e.g., HIPAA, SOC 2, GDPR).
Preferred Qualifications:
AWS certifications such as AWS Certified Solutions Architect – Associate , AWS Certified Security – Specialty , or AWS Certified Data Analytics – Specialty .
Experience with real-time data ingestion tools like AWS Kinesis or Kafka .
Familiarity with Data Governance frameworks and data lineage tools.
Experience with data visualization tools (e.g., AWS QuickSight, Tableau , Power BI ) and integrating them with cloud-based data lakes.
...Multiple Positions ALABASTER CITY SCHOOLS JOB DESCRIPTION JOB TITLE: CUSTODIAN QUALIFICATIONS: 1. Good general health. 2. Demonstrated aptitude for assigned responsibilities. 3. Such alternatives to the above qualification as the Board of Education...
...proficiency (or higher) in order to be a strong candidate? At least 8 years of experience in public policy, government, academia, or think tanks. An understanding of the landscape of AI and global risks, including expertise in one or more of: AI and cybersecurity, AI and...
...check out the following links: What You'll Do As a Senior Product Designer on the automotive team, youll create user-friendly, high-performance navigation experiences for mobile and in-car systems. Youll collaborate with engineers, design colleagues, and OEM partners...
St. Marys College of Maryland at Historic St. Marys City is accepting rsums for the position of Employer Relations and Events Coordinator. The Employer Relations and Events Coordinator is responsible for coordinating all recruiting and networking events within the...
...The Director* Global Clinical Operations provides leadership and overall strategic management of at least one Platform within Clinical Development Operations. This role interacts with and influences senior level management, external vendors, collaboration partners...