Skip to content

torch 🔗

Torch operations.

Buffer 🔗

Buffer(data: bytes | bytearray)

Bases: CType

Wrapper for the struct vaccel_torch_buffer C object.

Manages the creation and initialization of a C struct vaccel_torch_buffer and provides access to it through Python properties.

Inherits

CType: Abstract base class for defining C data types.

Parameters:

Name Type Description Default

data 🔗

 bytes | bytearray

The buffer data to be passed to the C struct.

 required
Source code in vaccel/ops/torch.py 
208
209
210
211
212
213
214
215
216
217
def __init__(self, data: bytes | bytearray):
    """Initializes a new `Buffer` object.

    Args:
        data: The buffer data to be passed to the C struct.
    """
    self._data = data
    self._c_data = None
    self._c_obj_ptr = None
    super().__init__()

c_size property 🔗

c_size: int

Returns the size of the object in bytes.

data property 🔗

data: bytes

The buffer data.

Returns:

Type Description
bytes

The data of the buffer.

size property 🔗

size: int

The buffer size.

Returns:

Type Description
int

The size of the buffer.

value property 🔗

value: CData

Returns the value of the underlying C struct.

Returns:

Type Description
CData

The dereferenced 'struct vaccel_torch_buffer`

Tensor 🔗

Tensor(dims: list[int], data_type: TensorType, data: list[Any])

Bases: CType

Wrapper for the struct vaccel_torch_tensor C object.

Manages the creation and initialization of a C struct vaccel_torch_tensor and provides access to it through Python properties.

Inherits

CType: Abstract base class for defining C data types.

Parameters:

Name Type Description Default

dims 🔗

 list[int]

The dims to be passed to the C struct.

 required

data_type 🔗

 TensorType

The data_type to be passed to the C struct.

 required

data 🔗

 list[Any]

The data to be passed to the C struct.

 required
Source code in vaccel/ops/torch.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
59
def __init__(self, dims: list[int], data_type: TensorType, data: list[Any]):
    """Initializes a new `Tensor` object.

    Args:
        dims: The dims to be passed to the C struct.
        data_type: The data_type to be passed to the C struct.
        data: The data to be passed to the C struct.
    """
    self._dims = dims
    self._data = data
    self._data_type = data_type
    self._c_obj_ptr = None
    self._c_obj_data = None
    super().__init__()

c_size property 🔗

c_size: int

Returns the size of the object in bytes.

data property 🔗

data: list | str

The tensor data.

Returns:

Type Description
list | str

The data of the tensor.

data_type property 🔗

data_type: TensorType

The tensor data type.

Returns:

Type Description
TensorType

The data type of the tensor.

dims property 🔗

dims: list[int]

The tensor dims.

Returns:

Type Description
list[int]

The dims of the tensor.

value property 🔗

value: CData

Returns the value of the underlying C struct.

Returns:

Type Description
CData

The dereferenced 'struct vaccel_torch_tensor`

empty classmethod 🔗

empty() -> Tensor

Initializes a new empty Tensor object.

The object a NULL pointer in place of the C struct.

Returns:

Type Description
Tensor

A new Tensor object
Source code in vaccel/ops/torch.py 
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
@classmethod
def empty(cls) -> "Tensor":
    """Initializes a new empty `Tensor` object.

    The object a NULL pointer in place of the C struct.

    Returns:
        A new `Tensor` object
    """
    inst = cls.__new__(cls)
    inst._c_obj_ptr = ffi.new("struct vaccel_torch_tensor **")
    inst._c_obj = inst._c_obj_ptr[0]
    inst._c_obj_ptr[0] = ffi.NULL
    inst._c_size = ffi.sizeof("struct vaccel_torch_tensor *")
    return inst

from_c_obj classmethod 🔗

from_c_obj(c_obj: CData) -> Tensor

Initializes a new Tensor object from an existing C struct.

Parameters:

Name Type Description Default

c_obj 🔗

 CData

A pointer to a struct vaccel_torch_tensor C object.

 required

Returns:

Type Description
Tensor

A new Tensor object
Source code in vaccel/ops/torch.py 
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
@classmethod
def from_c_obj(cls, c_obj: ffi.CData) -> "Tensor":
    """Initializes a new `Tensor` object from an existing C struct.

    Args:
        c_obj: A pointer to a `struct vaccel_torch_tensor` C object.

    Returns:
        A new `Tensor` object
    """
    type_str = ffi.getctype(ffi.typeof(c_obj))
    if type_str != "struct vaccel_torch_tensor *":
        msg = f"Expected 'struct vaccel_torch_tensor *', got '{type_str}'"
        raise TypeError(msg)

    inst = cls.__new__(cls)
    inst._c_obj = c_obj
    inst._c_size = ffi.sizeof(inst._c_obj)
    inst._dims = None
    inst._data = None
    inst._data_type = None
    inst._c_obj_ptr = None
    return inst

TorchMixin 🔗

Mixin providing Torch operations for a Session.

This mixin is intended to be used in combination with BaseSession and should not be instantiated on its own.

Intended usage

class Session(BaseSession, TorchMixin): ...

torch_jitload_forward 🔗

torch_jitload_forward(
    resource: Resource,
    in_tensors: list[Tensor],
    nr_out_tensors: int = 1,
    run_options: Buffer | None = None,
) -> list[Tensor]

Performs the Torch jitload forward operation.

Wraps the vaccel_torch_jitload_forward() C operation.

Parameters:

Name Type Description Default

resource 🔗

 Resource

A resource with the model to run.

 required

in_tensors 🔗

 list[Tensor]

The input tensors for the inference.

 required

nr_out_tensors 🔗

 int

The number of output tensors. Defaults to 1.

 1

run_options 🔗

 Buffer | None

The inference options.

 None

Returns:

Type Description
list[Tensor]

The output tensors

Raises:

Type Description
RuntimeError

If the Session is uninitialized.
FFIError

If the C operation fails.
Source code in vaccel/ops/torch.py 
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
def torch_jitload_forward(
    self,
    resource: Resource,
    in_tensors: list[Tensor],
    nr_out_tensors: int = 1,
    run_options: Buffer | None = None,
) -> list[Tensor]:
    """Performs the Torch jitload forward operation.

    Wraps the `vaccel_torch_jitload_forward()` C operation.

    Args:
        resource: A resource with the model to run.
        in_tensors: The input tensors for the inference.
        nr_out_tensors: The number of output tensors. Defaults to 1.
        run_options: The inference options.

    Returns:
        The output tensors

    Raises:
        RuntimeError: If the `Session` is uninitialized.
        FFIError: If the C operation fails.
    """
    if not self._c_ptr:
        msg = "Uninitialized session"
        raise RuntimeError(msg)

    run_options_ptr = (
        ffi.NULL if run_options is None else run_options._c_ptr
    )
    c_in_tensors = CList.from_ptrs(in_tensors)
    c_out_tensors = CList.from_ptrs([Tensor.empty()] * nr_out_tensors)

    ret = lib.vaccel_torch_jitload_forward(
        self._c_ptr,
        resource._c_ptr,
        run_options_ptr,
        c_in_tensors._c_ptr,
        len(c_in_tensors),
        c_out_tensors._c_ptr,
        len(c_out_tensors),
    )
    if ret != 0:
        raise FFIError(ret, "Torch jitload forward operation failed")

    return [Tensor.from_c_obj(t) for t in c_out_tensors.value]