Fixing WebSocket Silent Disconnects for Financial Market Data

Intro If you work with real-time financial tick data via WebSocket long connections, you’ve probably run into frustrating hidden issues: Connections show as online, but data stops flowing. Adding/removing trading symbols triggers connection storms. Minor network glitches break your data pipeline without obvious error logs. Today I’ll share a practical Python solution built around single-connection dynamic subscription and heartbeat timeout detection. We use alltick api as our example throughout this post. I’ll cover real production pain points, architecture, full runnable code, common bugs and optimization results. All code can be directly used in your projects. Real Production Pain Points & Background My team maintains real-time data pipelines for stocks, forex and cryptocurrencies. A core requirement is to add or remove monitored trading symbols on demand. At first, we created one independent WebSocket connection for each symbol. This simple approach caused a lot of problems in production: - Bulk symbol updates create frequent connection creation and destruction, leading to reconnection storms and high network load. - Ghost subscriptions: Data keeps coming even after you send an unsubscribe request. - False-alive sockets: The connection status stays connected, but tick data stops completely. Your analysis logic runs on invalid data. We refactored the architecture to use one single persistent WebSocket connection for all symbols, plus an independent thread for heartbeat monitoring. After the upgrade, all hidden connection issues are gone, and the system runs stably under high-frequency data traffic. 4 Common WebSocket Issues in Fintech Let’s break down the most frequent problems when using WebSocket for financial streaming. 1. Connection Flood & Reconnection Storms Rebuilding connections every time you update symbols causes endless handshakes and closures. It wastes system resources and makes the entire data stream unstable. 2. Undetectable False-Alive Sockets Network jitter or server rate limits won’t trigger a WebSocket close event. The link is actually broken, but logs show normal status. These silent failures are hard to debug. 3. Desynchronized Subscription States Rapid subscribe/unsubscribe requests cause race conditions. Your local symbol list no longer matches the server’s real subscription status, leaving many invalid subscriptions. 4. Missing Edge Case Validation Duplicate requests, empty symbol lists and invalid codes are not filtered in advance. Accumulated useless requests increase API parsing pressure. Complete Technical Solution What is Dynamic Subscription Dynamic subscription means updating your symbol list via API commands inside an existing WebSocket long connection. You never destroy the active connection. This is a standard pattern for high-frequency real-time data, different from traditional REST polling. Overall Architecture We split the solution into two decoupled modules to improve stability: - Dynamic Subscription: Manage all symbols on one long connection to avoid reconnection storms. - Heartbeat Monitoring: Run detection in a separate daemon thread. It works independently even when data processing logic is blocked. Common Use Cases & Rules Here are the most common scenarios you will meet during development, with API settings and validation rules. Initial bulk subscription for multiple symbols Creating a connection per symbol wastes resources. Use cmd_id=22004 + action=subscribe , and fill code with your symbol list like [NASDAQ:AAPL,BTCUSDT] . Validation: Only one WebSocket connection is created, and local storage syncs all symbol codes. Incrementally add new symbols New connections will cause stream instability. Use cmd_id=22004 + action=subscribe , set code to new symbols such as [EURUSD] . Validation: Original connection remains active; client only receives data from new symbols. Unsubscribe from specific symbols Redundant data still arrives after unsubscription. Use cmd_id=22004 + action=unsubscribe , write target symbols to the code field like [NASDAQ:AAPL] . Validation: Remove codes from local list; server stops pushing data for these symbols. Send duplicate subscription requests Repeated requests increase server load. When using cmd_id=22004 + action=subscribe for existing symbols like [BTCUSDT] , add client-side deduplication. Do not send duplicate requests to server. Send subscription with empty list Empty requests trigger runtime errors on both sides. Block empty code: [] requests locally before sending over the network. Full Python Code Implementation This code includes connection callbacks, local state management, heartbeat monitoring, dynamic subscription and data validation. We use Alltick API as the example, and it works in all standard Python environments. import websocket import json import time import threading # Manage subscription status and deduplication subscriptions = set() # Record the last heartbeat timestamp last_heartbeat_time = time.time() # Independent thread for heartbeat monitoring def heartbeat_monitor(): global last_heartbeat_time while True: current_ts = time.time() time_gap = current_ts - last_heartbeat_time # Timeout: 20s | Check interval: 5s if time_gap > 20: print("【Alert】Heartbeat timeout, connection link abnormal") # Extend: close connection / trigger reconnection / pause tasks break time.sleep(5) # Triggered when WebSocket connects successfully def on_open(ws): global subscriptions init_codes = ["NASDAQ:AAPL", "BTCUSDT"] subscriptions.update(init_codes) sub_req = { "cmd_id": 22004, "action": "subscribe", "code": init_codes } ws.send(json.dumps(sub_req)) print("Initial symbol subscription completed") # Handle incoming messages def on_message(ws, message): global last_heartbeat_time if not message: return try: data = json.loads(message) # Refresh heartbeat time if data.get("type") == "heartbeat": last_heartbeat_time = time.time() return # Process tick data and filter invalid values if data.get("type") == "tick": code = data.get("code", "") price = data.get("price", 0) open_24h = data.get("open_24h", 0) if not code or price <= 0 or open_24h <= 0: return print(f"Symbol {code} Latest Price: {price}") except Exception as e: print(f"Message parsing error: {str(e)}") # Connection error callback def on_error(ws, error): print(f"Connection error occurred: {error}") # Connection close callback def on_close(ws, close_status_code, close_msg): global subscriptions subscriptions.clear() print(f"Connection closed, status code: {close_status_code}") # Add new symbols to subscription def add_subscribe(ws, code_list): global subscriptions new_codes = [code for code in code_list if code not in subscriptions and code] if not new_codes: return subscriptions.update(new_codes) req = { "cmd_id": 22004, "action": "subscribe", "code": new_codes } ws.send(json.dumps(req)) # Unsubscribe selected symbols def cancel_subscribe(ws, code_list): global subscriptions remove_codes = [code for code in code_list if code in subscriptions and code] if not remove_codes: return for code in remove_codes: subscriptions.discard(code) req = { "cmd_id": 22004, "action": "unsubscribe", "code": remove_codes } ws.send(json.dumps(req)) if __name__ == "__main__": # Stock market WebSocket endpoint stock_wss_url = "wss://quote.alltick.co/quote-stock-b-ws-api?token=YOUR_TOKEN" # Forex & Crypto WebSocket endpoint common_wss_url = "wss://quote.alltick.co/quote-b-ws-api?token=YOUR_TOKEN" ws_app = websocket.WebSocketApp( common_wss_url, on_open=on_open, on_message=on_message, on_error=on_error, on_close=on_close ) # Start heartbeat monitor thread threading.Thread(target=heartbeat_monitor, daemon=True).start() # Built-in ping detection every 10 seconds ws_app.run_forever(ping_interval=10) Common Pitfalls & Fixes Here are the most frequent bugs we met during development and production, with simple solutions. Issue: High-volume tick data blocks callbacks and delay heartbeat detection Fix: Run heartbeat logic in a separate thread. Sort data by priority and process core data first.Issue: Network jitter causes false-alive sockets without close events Fix: Use heartbeat time gap to judge timeout. Close broken connections and reconnect orderly; pause data tasks during reconnection.Issue: Frequent add/remove requests lead to unsynchronized subscription states Fix: Use set to manage symbols, enable deduplication and limit request frequency.Issue: Wrong codes or empty strings cause silent subscription failure Fix: Add local validation for empty and invalid codes; auto-resubscribe for abnormal items. Feature Boundary - ✅ Supported: Dynamically add and remove symbols within one WebSocket connection. - ❌ Not supported: Cross-connection subscription sync, historical tick data query, custom commands other than cmd_id=22004 . Optimization Results After deploying this solution: - Lower resource usage: Single long connection reduces TCP handshake and teardown overhead for both client and server. - Stronger code robustness: Pre-validation blocks invalid requests and corrupted data. - Faster debugging: Heartbeat and subscription exceptions are fully logged. Silent disconnections are easy to locate. - More stable business: No more connection flutters, duplicate data or calculation errors for financial analysis. Discussion Have you ever dealt with WebSocket false-alive issues in fintech or real-time projects? What solutions do you use for connection keep-alive? Feel free to leave a comment below! 😊 Top comments (0)