Job Description
Job Description Pay rate: $80/hr - $100/hr on W2
Senior Data Engineer (GenAI / AWS)
Experience: 6 10 years
Cloud Platform: AWS
Primary Language: Python
Key Responsibilities:
1. Design and develop data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend.
2. Build automated document-processing workflows for extracting, transforming, and processing structured and unstructured data.
3. Implement document chunking, embeddings, vector indexing, and semantic search.
4. Develop and maintain vector databases and knowledge bases to support Retrieval-Augmented Generation (RAG).
5. Integrate knowledge bases with AI agents for contextual information retrieval.
6. Build AI agents using Claude Agent SDK and deploy them on Amazon Bedrock AgentCore Runtime.
7. Develop CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments.
8. Optimize data pipelines for scalability, reliability, security, and performance.
Must-have:
" Python, SQL
" AWS Step Functions, Lambda, S3
" Amazon Textract and Comprehend
" Vector databases and embeddings
" Document chunking and indexing
" RAG and knowledge-base integration
" Amazon Bedrock
" GitHub Actions CI/CD
Good-to-have:
" TypeScript
" AWS Glue
" Document AI / OCR
" OpenSearch, Pinecone, pgvector
" Advanced retrieval and reranking
" LangChain / LlamaIndex
" Claude Agent SDK
" Terraform / AWS CDK
" AgentCore Runtime