Skip to content

Step into from Python to C++ doesn't work if arguments are unpacked into function call #7129

New issue
New issue
Closed
Closed
Step into from Python to C++ doesn't work if arguments are unpacked into function call#7129
Assignees
bschnurr
Labels
bug

Description

@vsfeedback
Visual Studio Feedback Bot (vsfeedback)
opened on Aug 18, 2022

This issue has been moved from a ticket on Developer Community.

[severity:It bothers me. A fix would be nice]
It seems that in a mixed-mode Python/C++ debugging session, while debugging a piece of Python code, stepping into a function whose implementation is in C++, this does not work if the arguments are unpacked into the function call.

Example:

fast_foo(1, 2, 3)  # works, I manage to step into the C++ code

args = (1, 2, 3)
fast_foo(*args)  # does not work, it instead steps over the function call

A work around seems to be to either set a break point in the C++ code, or to change the calling code:

args = (1, 2, 3)
fast_foo.__call__(*args)

For some reason, using the __call__ method seems to not suffer from the problem. However, both of these work arounds are not really satisfactory.

Original Comments

(no comments)

Original Solutions

(no solutions)

Activity

  1. github-actions
    added
    needs triage
    on Aug 18, 2022
  2. Pavel Minaev (int19h)
    added
    bug
    and removed
    needs triage
    on Aug 30, 2022

  3. int19h commented on Aug 30, 2022

    @int19h
    Pavel Minaev (int19h)
    on Aug 30, 2022
    Contributor

    Likely due to code not keeping up with various possible step-in code flows in newer versions of Python.

  4. int19h commented on Aug 30, 2022

    @int19h
    Pavel Minaev (int19h)
    on Aug 30, 2022
    Contributor

    Relevant code parts:

    PTVS/Python/Product/Debugger.Concord/TraceManagerLocalHelper.cs

    Lines 220 to 223 in 421d001

    // A step-in gate is a function inside the Python interpreter or one of the libaries that may call out
    // to native user code such that it may be a potential target of a step-in operation. For every gate,
    // we record its address in the process, and create a breakpoint. The breakpoints are initially disabled,
    // and only get enabled when a step-in operation is initiated - and then disabled again once it completes.

    PTVS/Python/Product/Debugger.Concord/TraceManagerLocalHelper.cs

    Lines 539 to 813 in 421d001

    [StepInGate]
    public void call_function(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    int oparg = cppEval.EvaluateInt32(useRegisters ? "@rdx" : "oparg");
    int na = oparg & 0xff;
    int nk = (oparg >> 8) & 0xff;
    int n = na + 2 * nk;
    ulong func = cppEval.EvaluateUInt64(
    "*((*(PyObject***){0}) - {1} - 1)",
    useRegisters ? "@rcx" : "pp_stack",
    n);
    var obj = PyObject.FromAddress(process, func);
    ulong ml_meth = cppEval.EvaluateUInt64(
    "((PyObject*){0})->ob_type == &PyCFunction_Type ? ((PyCFunctionObject*){0})->m_ml->ml_meth : 0",
    func);
    _owner.OnPotentialRuntimeExit(thread, ml_meth);
    }
    [StepInGate]
    public void PyCFunction_Call(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    ulong ml_meth = cppEval.EvaluateUInt64(
    "((PyObject*){0})->ob_type == &PyCFunction_Type ? ((PyCFunctionObject*){0})->m_ml->ml_meth : 0",
    useRegisters ? "@rcx" : "func");
    _owner.OnPotentialRuntimeExit(thread, ml_meth);
    }
    [StepInGate]
    public void getset_get(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string descrVar = useRegisters ? "((PyGetSetDescrObject*)@rcx)" : "descr";
    ulong get = cppEval.EvaluateUInt64(descrVar + "->d_getset->get");
    _owner.OnPotentialRuntimeExit(thread, get);
    }
    [StepInGate]
    public void getset_set(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string descrVar = useRegisters ? "((PyGetSetDescrObject*)@rcx)" : "descr";
    ulong set = cppEval.EvaluateUInt64(descrVar + "->d_getset->set");
    _owner.OnPotentialRuntimeExit(thread, set);
    }
    [StepInGate(HasMultipleExitPoints = true)]
    public void type_call(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string typeVar = useRegisters ? "((PyTypeObject*)@rcx)" : "type";
    ulong tp_new = cppEval.EvaluateUInt64(typeVar + "->tp_new");
    _owner.OnPotentialRuntimeExit(thread, tp_new);
    ulong tp_init = cppEval.EvaluateUInt64(typeVar + "->tp_init");
    _owner.OnPotentialRuntimeExit(thread, tp_init);
    }
    [StepInGate]
    public void PyType_GenericNew(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string typeVar = useRegisters ? "((PyTypeObject*)@rcx)" : "type";
    ulong tp_alloc = cppEval.EvaluateUInt64(typeVar + "->tp_alloc");
    _owner.OnPotentialRuntimeExit(thread, tp_alloc);
    }
    [StepInGate]
    public void PyObject_Print(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string opVar = useRegisters ? "((PyObject*)@rcx)" : "op";
    ulong tp_print = cppEval.EvaluateUInt64(opVar + "->ob_type->tp_print");
    _owner.OnPotentialRuntimeExit(thread, tp_print);
    }
    [StepInGate]
    public void PyObject_GetAttrString(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_getattr = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_getattr");
    _owner.OnPotentialRuntimeExit(thread, tp_getattr);
    }
    [StepInGate]
    public void PyObject_SetAttrString(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_setattr = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_setattr");
    _owner.OnPotentialRuntimeExit(thread, tp_setattr);
    }
    [StepInGate]
    public void PyObject_GetAttr(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_getattr = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_getattr");
    _owner.OnPotentialRuntimeExit(thread, tp_getattr);
    ulong tp_getattro = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_getattro");
    _owner.OnPotentialRuntimeExit(thread, tp_getattro);
    }
    [StepInGate]
    public void PyObject_SetAttr(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_setattr = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_setattr");
    _owner.OnPotentialRuntimeExit(thread, tp_setattr);
    ulong tp_setattro = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_setattro");
    _owner.OnPotentialRuntimeExit(thread, tp_setattro);
    }
    [StepInGate]
    public void PyObject_Repr(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_repr = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_repr");
    _owner.OnPotentialRuntimeExit(thread, tp_repr);
    }
    [StepInGate]
    public void PyObject_Hash(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_hash = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_hash");
    _owner.OnPotentialRuntimeExit(thread, tp_hash);
    }
    [StepInGate]
    public void PyObject_Call(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string funcVar = useRegisters ? "((PyObject*)@rcx)" : "func";
    ulong tp_call = cppEval.EvaluateUInt64(funcVar + "->ob_type->tp_call");
    _owner.OnPotentialRuntimeExit(thread, tp_call);
    }
    [StepInGate]
    public void PyObject_Str(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    ulong tp_str = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_str");
    _owner.OnPotentialRuntimeExit(thread, tp_str);
    }
    [StepInGate(MaxVersion = PythonLanguageVersion.V27, HasMultipleExitPoints = true)]
    public void do_cmp(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    string wVar = useRegisters ? "((PyObject*)@rdx)" : "w";
    ulong tp_compare1 = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_compare");
    _owner.OnPotentialRuntimeExit(thread, tp_compare1);
    ulong tp_richcompare1 = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare1);
    ulong tp_compare2 = cppEval.EvaluateUInt64(wVar + "->ob_type->tp_compare");
    _owner.OnPotentialRuntimeExit(thread, tp_compare2);
    ulong tp_richcompare2 = cppEval.EvaluateUInt64(wVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare2);
    }
    [StepInGate(MaxVersion = PythonLanguageVersion.V27, HasMultipleExitPoints = true)]
    public void PyObject_RichCompare(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    string wVar = useRegisters ? "((PyObject*)@rdx)" : "w";
    ulong tp_compare1 = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_compare");
    _owner.OnPotentialRuntimeExit(thread, tp_compare1);
    ulong tp_richcompare1 = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare1);
    ulong tp_compare2 = cppEval.EvaluateUInt64(wVar + "->ob_type->tp_compare");
    _owner.OnPotentialRuntimeExit(thread, tp_compare2);
    ulong tp_richcompare2 = cppEval.EvaluateUInt64(wVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare2);
    }
    [StepInGate(MinVersion = PythonLanguageVersion.V33, HasMultipleExitPoints = true)]
    public void do_richcompare(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string vVar = useRegisters ? "((PyObject*)@rcx)" : "v";
    string wVar = useRegisters ? "((PyObject*)@rdx)" : "w";
    ulong tp_richcompare1 = cppEval.EvaluateUInt64(vVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare1);
    ulong tp_richcompare2 = cppEval.EvaluateUInt64(wVar + "->ob_type->tp_richcompare");
    _owner.OnPotentialRuntimeExit(thread, tp_richcompare2);
    }
    [StepInGate]
    public void PyObject_GetIter(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string oVar = useRegisters ? "((PyObject*)@rcx)" : "o";
    ulong tp_iter = cppEval.EvaluateUInt64(oVar + "->ob_type->tp_iter");
    _owner.OnPotentialRuntimeExit(thread, tp_iter);
    }
    [StepInGate]
    public void PyIter_Next(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string iterVar = useRegisters ? "((PyObject*)@rcx)" : "iter";
    ulong tp_iternext = cppEval.EvaluateUInt64(iterVar + "->ob_type->tp_iternext");
    _owner.OnPotentialRuntimeExit(thread, tp_iternext);
    }
    [StepInGate]
    public void builtin_next(DkmThread thread, ulong frameBase, ulong vframe, bool useRegisters) {
    var process = thread.Process;
    var cppEval = new CppExpressionEvaluator(thread, frameBase, vframe);
    string argsVar = useRegisters ? "((PyTupleObject*)@rdx)" : "((PyTupleObject*)args)";
    ulong tp_iternext = cppEval.EvaluateUInt64(argsVar + "->ob_item[0]->ob_type->tp_iternext");
    _owner.OnPotentialRuntimeExit(thread, tp_iternext);
    }

    So basically there's some code path that handles the CALL_... bytecode opcodes that either doesn't go through any of these, or if it does, it doesn't set a breakpoint on some of the potential destinations.

  5. Stella Huang (StellaHuang95)
    assigned
    Bill Schnurr (bschnurr)
    on Dec 12, 2023

  6. cwebster-99 commented on Aug 14, 2024

    @cwebster-99
    Courtney Webster (cwebster-99)
    on Aug 14, 2024
    Member

    Thank you for your report! While we recognize this bug is a limitation, due to our limited resources, we may not be able to prioritize this in the near term. We are going to move this to discussion and give the community 60 days to provide 7 👍 upvotes to gauge general interest. If there's enough upvotes then we will consider incorporating this in an upcoming sprint. If there's unfortunately not enough upvotes then we will close this issue. Thanks!

  7. microsoft
    locked and limited conversation to collaborators on Aug 14, 2024
  8. Courtney Webster (cwebster-99)
    converted this issue into a discussion #7981 on Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions