For PyTorch, `+=` is interpreted as an in-place operation

My guess is that it operates in place with no memory allocations or copying.

Not exactly:

    >>> def f(x): x += 1
    ... 
    >>> def g(x): x = x + 1
    ... 
    >>> dis.dis(f)
      1           0 LOAD_FAST                0 (x)
                  3 LOAD_CONST               1 (1)
                  6 INPLACE_ADD         
                  7 STORE_FAST               0 (x)
                 10 LOAD_CONST               0 (None)
                 13 RETURN_VALUE        
    >>> dis.dis(g)
      1           0 LOAD_FAST                0 (x)
                  3 LOAD_CONST               1 (1)
                  6 BINARY_ADD          
                  7 STORE_FAST               0 (x)
                 10 LOAD_CONST               0 (None)
                 13 RETURN_VALUE