Automation QA Expert (AI/ML Focus)

Transform quality assurance from a manual bottleneck into an automated accelerator. You’ll build the automated testing infrastructure that enables rapid, safe deployments of AI services and ML models to production. By creating comprehensive unit tests, integration tests, and CI/CD test pipelines, you’ll dramatically reduce manual testing effort while increasing quality and release confidence. Your automation frameworks will protect critical banking flows 24/7, catch regressions before they reach customers, and provide audit-ready evidence of quality—making you essential to both velocity and governance.

General duties:

Design and implement automated testing strategies across unit, integration, and end-to-end levels for AI/ML services, APIs, and platform components.

Increase unit test coverage for Python services and ML model code; establish coverage targets and track metrics.

Create automated integration tests for cross-system flows involving ML & AI platforms and external systems (UFM, Remedy, SmartVista, etc.).

Build and maintain CI/CD test pipelines integrated with Azure DevOps, GitHub Actions, or similar platforms.

Develop testing frameworks and reusable test utilities for AI Agents, ML model validation, and API contract testing.

Implement automated API testing using tools like pytest, Postman/Newman, or REST Assured.

Create automated data quality and integrity checks for ML pipelines and streaming data flows.

Collaborate with Software Engineers, ML Engineers, and Data Engineers to establish testability standards and best practices.

Maintain test automation documentation, reports, and dashboards to track quality metrics.

Perform code reviews on test code and promote test automation culture across teams.

Support manual QA and developers by automating high-priority regression scenarios .

Requirements:

Must have

4+ years of experience in test automation or software development with strong testing focus.

Proficiency in Python or similar programming language.

Experience with API test automation.

Solid understanding of software testing principles and test design patterns.

Strong SQL skills for data validation and integrity testing.

Verbal and written English at Upper Intermediate level or above.

Nice to have

Expert-level proficiency in Python and testing frameworks (pytest, unittest, or similar).

Strong experience with API test automation using tools like Postman/Newman, REST Assured, or Python requests library.

Proven track record of building and maintaining CI/CD test pipelines (Azure DevOps, GitHub Actions, Jenkins, GitLab CI).

Experience with integration testing and end-to-end test automation frameworks (Selenium, Playwright, Cypress, or similar).

Understanding of testing pyramid and test coverage strategies.

Experience with test coverage tools and quality metrics reporting.

Proficiency in Git, version control workflows, and code review practices.

Ability to write clean, maintainable test code following best practices.

Excellent collaboration skills for working with developers, QA, and DevOps teams.

Experience with ML/AI testing, including model validation and LLM evaluation frameworks.

Knowledge of performance testing tools (Locust, JMeter, k6).

Experience with contract testing frameworks (Pact, Spring Cloud Contract).

Familiarity with Azure services and cloud-native testing approaches.

Experience in fintech, banking, or regulated industries.

Knowledge of test data management and synthetic data generation ISTQB® Test Automation Engineer or similar certification.

What we offer to you:

Excellent opportunities for professional and career development in one of Bulgaria’s leading banks;

Food vouchers in the amount of 102.26 EUR per month;

25 paid holiday leave;

Additional Health Insurance;

Favorable conditions for housing and mortgage lending, as well as for bank products and services;

Annual bonus scheme depending on the achieved results;

Preferential conditions for Multisport / CoolFit card;

Great location;

Discounts in various companies;

Professional trainings for specific knowledge and skills (internal academies);

Refer a Friend Bonus.

If you are challenged by this opportunity, we would be glad to review your application.