Asserting exception or return value (raises_or_returns(...))

[plotski]

I frequently parametrize my tests to call the same function with different arguments and the
expected results for those arguments. The result can either be a return value or an exception.

This is a very basic example of what this looks like:

import pytest, re

def add(integers, *, max_sum):
    result = sum(integers)
    if result > max_sum:
        raise RuntimeError(f'Sum is too large: {result}')
    return result

@pytest.mark.parametrize(
    argnames='integers, max_sum, exp_result',
    argvalues=(
        (
            (1, 2, 3),
            10,
            6,
        ),
        (
            (1, 2, 9),
            10,
            RuntimeError('Sum is too large: 12'),
        ),
    ),
)
def test_sum(integers, max_sum, exp_result):
    if isinstance(exp_result, Exception):
        with pytest.raises(type(exp_result), match=rf'^{re.escape(str(exp_result))}$'):
            add(integers, max_sum=max_sum)
    else:
        return_value = add(integers, max_sum=max_sum)
        assert return_value == exp_result

Writing lots of test functions like this is very tedious. But we can put everything in a separate
function:

def raises_or_returns(exp_result, function, *args, **kwargs):
    if isinstance(exp_result, Exception):
        with pytest.raises(type(exp_result), match=rf'^{re.escape(str(exp_result))}$'):
            function(*args, **kwargs)
    else:
        return_value = function(*args, **kwargs)
        assert return_value == exp_result

And now my test function has only 2 lines instead of 7:

def test_sum(integers, max_sum, exp_result):
    raises_or_returns(exp_result, add, integers, max_sum=max_sum)

In principle, would something like raises_or_returns be a good addition to the pytest helpers (like pytest.raises)?
Or maybe there there is a better solution to my problem that I've missed?

[RonnyPfannschmidt]

Similar apis have been suggested ,it's a wrong abstraction.

2 helpers are needed, obe that casts a function call result into a future like result and one that expresses matching exceptions or results

[wimglenn]

I don't think it is a good addition, too much scope creep here and cramming too much logic into test code. It is also perfectly valid for a function to return an exception, not raise it, and this case would be hard to distinguish.

Better alternative is to create two different parametrize tests, one for the returning results and one for the raising cases.

[plotski]

I agree that testing success and failure in separate test functions can be nicer to read, but that would *increase* the amount of boilerplate code, and my concern is to reduce it.

[The-Compiler]

The docs already have something on that: Parametrizing tests - pytest documentation

But I agree with what's been said, I consider this an antipattern and would be -1 on adding a (rather tricky to understand) helper like that.

[plotski]

That looks like a viable solution for me. Thanks!