QA ENGINEER with Conversational AI or Gen AI Experience

Job Description

Hi,

I hope you are doing well!

We have an opportunity for QA ENGINEER with Conversational AI or GenAI Experience with one of our clients for Hartford, CT.

Please see the job details below and let me know if you would be interested in this role.

If interested, please send me a copy of your resume, contact details, availability, and a good time to connect with you.

Title: QA ENGINEER with Conversational AI or GenAI Experience

Location: Hartford, CT- Hybrid

Terms: Long Term Contract

Job Details:

Position Summary

The Aetna IT Delivery organization, part of CVS Health’s Data, Digital, Analytics, Technology (DDAT) division, is seeking a highly skilled Quality Assurance Automation Engineer to design, develop, and maintain automated testing solutions for virtual assistants (agents) to support members and providers by accessing timely, accurate health insurance answers across digital channels.

The ideal candidate will have strong experience in Behavior Driven Development and Test Driven Development methodologies, automation scripting, and healthcare domain knowledge.

Preferred Qualifications

• Conversational AI Testing: Hands-on experience or strong exposure to testing chatbots and voice-based virtual assistants across multiple platforms.
• AI Model Evaluation: Familiarity with key evaluation metrics such as accuracy, precision, recall, and F1-score for assessing AI model performance.
• Experience with Agentic IDEs: Familiarity with AI-powered development environments (e.g., Windsurf, Cursor, or similar) for creating, maintaining, and optimizing automated test scripts.
• Experience with API testing (REST, SOAP)
• Knowledge of cloud platforms (AWS, GCP, or IBM Cloud)

Required Qualifications

• 3+ years of experience in QA automation and software testing.
• Strong technical expertise in core programming concepts (e.g., Java OOPs, Data Structures) with the ability to adopt and work with multiple languages such as Java, JavaScript, or Python.
• Hands-on experience implementing and enhancing test automation frameworks, including designing scalable and maintainable solutions.
• Hands-on experience with automation tools including Hammer On Demand and Selenium
• Strong understanding of SDLC, Agile methodologies, and QA principles.
• Experience with CI/CD tools and version control systems (e.g., Git).
• Frameworks & Methodologies:

BDD (e.g., Cucumber, SpecFlow)

TDD

• Testing Expertise: Functional, Regression, Integration, and Performance Testing, including proficiency in backend API test automation and database testing
• Healthcare Domain Knowledge: Familiarity with HIPAA compliance and healthcare workflows
• Exposure to GenAI and AI-driven tools utilized for test automation, with an understanding of how these can be integrated into QA processes
• Strong analytical and problem-solving skills
• Excellent communication and collaboration abilities.
• Ability to clearly articulate technical concepts and demonstrate practical problem-solving skills during discussions.

Key Responsibilities

Test Automation Development:

• Design, develop, and maintain automated test scripts using industry-standard tools and frameworks (e.g., Selenium, Cypress, Playwright, Appium).
• Build and enhance automation frameworks to support functional, regression, and performance testing.

Quality Assurance:

• Collaborate with developers and QA analysts to understand application functionality and define test coverage.
• Execute automated test suites and analyze results to identify defects and ensure compliance with quality standards.

Continuous Integration & Delivery (CI/CD):

• Integrate automated tests into CI/CD pipelines using tools like Jenkins, GitLab CI, or Azure DevOps.
• Monitor and maintain test execution in CI environments to ensure timely feedback.

Defect Management:

• Log, track, and verify defects using tools such as JIRA or Azure DevOps.
• Work with development teams to reproduce issues and validate fixes.

Documentation & Reporting:

• Prepare detailed test plans, test cases, and automation scripts documentation.
• Generate and present test execution reports to stakeholders.

Process Improvement:

• Identify opportunities to improve automation coverage and efficiency.
• Stay updated with emerging trends, tools, and best practices in test automation.

Thank you!

Amit Jha

Senior Recruiter | BeaconFire Inc.

📧 amitj@beaconfireinc.com