feat(executorch): share one TensorRT engine between loads of the same program

[shoumikhin]

What is wrong today

A robot with two arms often runs the same policy twice in one process, from one .pte file. Each arm loads it:

method = Runtime.get().load_program(path).load_method("forward")

Each load deserializes a new TensorRT engine, which holds the weights, so the second arm pays for a second copy: about 800 MiB for SmolVLA on an 8 GB Jetson Orin Nano, where the CPU and GPU share one memory.

What this change does

Loads of the same engine bytes, device and weight streaming budget now share one engine. Each load keeps its own execution context and buffers; the last one frees the engine.

• TensorRT cannot change the budget once a context exists, so the first load sets it. Another budget gets its own engine.
• Engines are matched by a hash of their bytes and size, since the bytes are freed after loading. This trusts the program, as the README asks.
• If two racing loads both deserialize and the second copy does not fit, that load shares the first engine instead of failing.
• Setting the backend option use_shared_engines (on by default) to false turns sharing off for later loads. Only C++ can set it: the ExecuTorch Python bindings do not expose set_option.

The first commit only moves the budget code into a function.

Cost

Every load now hashes the engine bytes: about 0.15 s per GB on a Jetson AGX Thor, 0.25 s on the Orin Nano. Inference speed does not change.

Results

On the Orin Nano, a second load of the same SmolVLA program costs about 120 MiB instead of 800 MiB. Outputs are unchanged.

What was tested

• New test_shared_engines: sharing, each part of the key, freeing, budgets, a failed load, no second deserialize on a match, the map lock while TensorRT logs, concurrent loads, two streams, dynamic shapes, the default and the option parser.
• Of six deliberately broken backends, four fail a case: a key ignoring the bytes, a skipped first lookup, a failed load keeping the engine, and the map lock held during loading. A key ignoring the device needs a two-GPU case. A racing load that does not look again shows up only in a separate race under a memory cap.
• The existing delegate tests give the same results before and after. The new test passed 20 runs in a row on a Jetson AGX Thor.