I was trying to use the QAEvaluator w/ a reasoning model w/ the is_reasoning_model: true parameter and ran into the following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.", 'type': 'invalid_request_error', 'param': 'max_tokens', 'code': 'unsupported_parameter'}}

This doesn't happen for individual evaluators that the QAEvaluator makes use of. It seems to be due to the fact that the QAEvaluator does not pass through the is_reasoning_model for the evaluators under the hood > https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_qa/_qa.py#L104-L111