Describe the bug
make_pair_ in src/relay_server.rs looks up the waiting peer and registers a new one under two separate acquisitions of the PEERS mutex:
let mut peer = PEERS.lock().await.remove(&rf.uuid); // lock #1
if let Some(peer) = peer.as_mut() { /* pair + relay */ }
else {
PEERS.lock().await.insert(rf.uuid.clone(), Box::new(stream)); // lock #2
sleep(30.).await;
PEERS.lock().await.remove(&rf.uuid);
}
If both legs of a relay (same uuid) arrive between those two acquisitions, both miss and both take the else branch. The second insert replaces (and drops) the first stream, so neither leg is ever paired: one side is closed immediately and the other waits the full 30 s before being dropped. The client sees a failed relay connection.
Describe the environment
- Install environment: docker (official
rustdesk/rustdesk-server image, hbbs + hbbr, -k _, ENCRYPTED_ONLY=1)
- Version in production: 1.1.14. Reproduced on current
master (a7736be) built from source, unmodified.
- Clients: RustDesk 1.4.7 (Windows).
How to Reproduce the bug
- Build
hbbr from master and run it on a spare port: hbbr -p 31317 -k "".
- From a non-loopback address of that host (loopback connections go to hbbr's admin console), open two TCP connections, send the same
RequestRelay frame (same uuid) on both back-to-back, then send a byte on one leg and expect it on the other. Script below.
- Some pairs are never relayed. Each lost uuid shows two
New relay request <uuid> lines and no got paired.
Results, 200 simultaneous pairs per run (a control run with the two legs 0.2 s apart pairs 20/20):
| build |
run 1 |
run 2 |
master a7736be, unmodified |
177/200 |
176/200 |
master + patch below |
200/200 |
200/200 |
All 47 unpaired uuids in the unmodified runs show the double New relay request signature. 1.1.14 behaves the same (125/150 unmodified, 150/150 patched).
Expected behavior
Both legs of a relay are always paired, regardless of how close together they arrive.
Additional context
In production (1.1.14) we found 11 such uuids in about four months, mostly between legs with very low latency to the relay (same LAN). On the target side the RustDesk log shows create_relay requested ... uuid: <uuid> with no following Connection opened, i.e. the connection attempt silently failed.
Open PR #642 restructures this block but still does the remove and the insert under separate PEERS acquisitions, so it does not close this race.
Suggested fix: do the check and the insert under one lock.
diff --git a/src/relay_server.rs b/src/relay_server.rs
index de1a7ea..609f0cb 100644
--- a/src/relay_server.rs
+++ b/src/relay_server.rs
@@ -468,7 +468,22 @@ async fn make_pair_(stream: impl StreamTrait, addr: SocketAddr, key: &str, limit
return;
}
if !rf.uuid.is_empty() {
- let mut peer = PEERS.lock().await.remove(&rf.uuid);
+ // Check-and-insert under one lock: with separate lock acquisitions, two
+ // legs of the same uuid arriving together can both miss and never pair.
+ let mut peers = PEERS.lock().await;
+ let mut peer = peers.remove(&rf.uuid);
+ if peer.is_none() {
+ log::info!("New relay request {} from {}", rf.uuid, addr);
+ peers.insert(rf.uuid.clone(), Box::new(stream));
+ drop(peers);
+ sleep(30.).await;
+ PEERS.lock().await.remove(&rf.uuid);
+ return;
+ }
+ drop(peers);
if let Some(peer) = peer.as_mut() {
log::info!("Relayrequest {} from {} got paired", rf.uuid, addr);
let id = format!("{}:{}", addr.ip(), addr.port());
@@ -485,11 +500,6 @@ async fn make_pair_(stream: impl StreamTrait, addr: SocketAddr, key: &str, limit
log::info!("Relay of {} closed", addr);
}
USAGE.write().await.remove(&id);
- } else {
- log::info!("New relay request {} from {}", rf.uuid, addr);
- PEERS.lock().await.insert(rf.uuid.clone(), Box::new(stream));
- sleep(30.).await;
- PEERS.lock().await.remove(&rf.uuid);
}
}
}
Repro script (Python 3, stdlib only)
# A/B harness for the hbbr pair race. Usage: hbbr_pair_race_test.py <port> <pairs> <gap_s> <non-loopback-host-ip>
import socket, sys, uuid, time
PORT, N, GAP, HOST = int(sys.argv[1]), int(sys.argv[2]), float(sys.argv[3]), sys.argv[4]
def frame(u):
inner = b'\x12' + bytes([len(u)]) + u # RequestRelay.uuid (field 2)
msg = b'\x92\x01' + bytes([len(inner)]) + inner # RendezvousMessage.request_relay (field 18)
return bytes([len(msg) << 2]) + msg # hbb_common 1-byte length header
ok = 0
for _ in range(N):
u = str(uuid.uuid4()).encode()
a = socket.create_connection((HOST, PORT)); b = socket.create_connection((HOST, PORT))
for s in (a, b): s.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
time.sleep(0.02); f = frame(u); a.send(f)
if GAP: time.sleep(GAP)
b.send(f); time.sleep(0.1)
try:
a.send(b'ping-from-a'); b.settimeout(1); ok += (b.recv(64) == b'ping-from-a')
except Exception:
pass
a.close(); b.close(); time.sleep(0.05)
print(f'port {PORT} gap={GAP}: paired+relayed {ok}/{N}', flush=True)
Usage: python3 hbbr_pair_race_test.py <port> <pairs> <gap_seconds> <non-loopback-host-ip>, e.g. python3 hbbr_pair_race_test.py 31317 200 0 10.0.0.5.
Describe the bug
make_pair_insrc/relay_server.rslooks up the waiting peer and registers a new one under two separate acquisitions of thePEERSmutex:If both legs of a relay (same
uuid) arrive between those two acquisitions, both miss and both take theelsebranch. The secondinsertreplaces (and drops) the first stream, so neither leg is ever paired: one side is closed immediately and the other waits the full 30 s before being dropped. The client sees a failed relay connection.Describe the environment
rustdesk/rustdesk-serverimage, hbbs + hbbr,-k _,ENCRYPTED_ONLY=1)master(a7736be) built from source, unmodified.How to Reproduce the bug
hbbrfrommasterand run it on a spare port:hbbr -p 31317 -k "".RequestRelayframe (same uuid) on both back-to-back, then send a byte on one leg and expect it on the other. Script below.New relay request <uuid>lines and nogot paired.Results, 200 simultaneous pairs per run (a control run with the two legs 0.2 s apart pairs 20/20):
mastera7736be, unmodifiedmaster+ patch belowAll 47 unpaired uuids in the unmodified runs show the double
New relay requestsignature. 1.1.14 behaves the same (125/150 unmodified, 150/150 patched).Expected behavior
Both legs of a relay are always paired, regardless of how close together they arrive.
Additional context
In production (1.1.14) we found 11 such uuids in about four months, mostly between legs with very low latency to the relay (same LAN). On the target side the RustDesk log shows
create_relay requested ... uuid: <uuid>with no followingConnection opened, i.e. the connection attempt silently failed.Open PR #642 restructures this block but still does the
removeand theinsertunder separatePEERSacquisitions, so it does not close this race.Suggested fix: do the check and the insert under one lock.
Repro script (Python 3, stdlib only)
Usage:
python3 hbbr_pair_race_test.py <port> <pairs> <gap_seconds> <non-loopback-host-ip>, e.g.python3 hbbr_pair_race_test.py 31317 200 0 10.0.0.5.