Twitch Accepts Invalid Keys and Silently Discards Them - The Hidden Pitfalls of Integration and a 41-Second Recovery

📝 Originally published (in Japanese) at forge.workstyle.tech. Building an AI Avatar Live Streaming System That Runs Unattended When building an AI avatar live streaming system that runs unattended, the development focus shifts from a typical application. If the system crashes during unmonitored hours, no one is there to fix it. This means that instead of just "working well," the key to quality is making sure the system "understands when it breaks and can recover on its own." This article summarizes the pitfalls encountered while getting such a system to run continuously on Twitch, broken down into three layers: - Authentication Layer: OAuth for EventSub and metadata manipulation. The constraints of manual human steps and tokens that aren’t immutable. - Transmission Layer: Even when ffmpeg is sending video, the channel doesn’t go live. Twitch silently accepts invalid keys. - Fault Tolerance Layer: There are three ways the server can die during a stream, one of which leaves the video running silently. All of these issues share a common pattern: "Everything looks normal from the sender’s side." This makes them especially tricky. Let’s go through them one by one. Where OAuth Actually Becomes Necessary First, to just read chat, an anonymous IRC connection is sufficient-no app registration required. That part was straightforward. OAuth became necessary when we wanted to do two things: - Use EventSub to receive bits (cheers), subscriptions, and raids, and have the avatar react. - Use the Helix API to automatically set the stream title per episode. Only these two operations require authorization. The overall flow looks like this: 1. Register an app in the Developer Console (Confidential type) 2. Obtain client_id / client_secret 3. Open the authorization URL in a browser and authorize as the channel owner 4. Extract the authorization code (code) from the redirect URL 5. Exchange the code for an access token and refresh token via the token endpoint 6. Save the refresh token on the server Only steps 3 and 4 require human interaction; the rest can be automated. Below are the specific pain points we encountered during this process. App Type and Redirect URI When registering the app, you must select a type. If your server holds the client_secret and performs token exchange, choose Confidential. Choosing the wrong type later means the secret can’t be used for token exchange, and you’ll have to recreate the app. The redirect URI is only needed to complete authorization, so something like http://localhost:3000 is fine. You don’t even need to run a server to receive it-just copy the code from the address bar after authorization. Request All Required Scopes in a Single Authorization If scopes are missing, you’ll have to restart the authorization process. The three scopes we needed were: | Scope | Purpose | |---|---| bits:read | Subscribe to bits (cheers) events | channel:read:subscriptions | Subscribe to subscription events | channel:manage:broadcast | Set stream metadata like the title | Adding scopes later requires the user to open a browser again. List all required functionality upfront and request all necessary scopes in one authorization. Use force_verify to Avoid Authorizing the Wrong Account We added force_verify=true to the authorization URL. Without it, if you’re already logged into the browser, authorization might complete without a confirmation screen, especially if you have both a personal account and a character account. This can lead to accidentally authorizing the wrong account. With force_verify=true , a confirmation screen always appears, making it clear which account you’re authorizing. Authorization Codes Expire in Minutes This was the most nerve-wracking issue. Authorization codes (code ) expire in just a few minutes. If the flow involves human steps-opening a browser, copying the code, passing it to the server, and exchanging it-the code may already be dead by the time you receive it. The fix is simple: prepare the token exchange process in advance and execute it the moment the code arrives. We pre-assembled the exchange command and waited. As soon as the code came in, we executed it immediately, and it worked on the first try. After Obtaining Credentials, Verify Who You’re Authorized As Once you get the token, hit the validation endpoint (/oauth2/validate ) to inspect its contents. curl -H "Authorization: OAuth " https://id.twitch.tv/oauth2/validate Check the returned login (username) and the list of scopes. Are you authorized as the expected channel owner? Are all required scopes present? Skipping this step makes it impossible to distinguish between "insufficient scopes," "wrong account," and "implementation error" when EventSub subscriptions fail. Immediately after obtaining credentials, verify what they represent. Operational Pitfall: Refresh Tokens Can Be Replaced This issue became apparent only after implementation and is worth sharing. Twitch may replace the refresh token itself when updating tokens. A naive implementation looks like this: - On startup, read the saved refresh token - Use it to refresh the access token - A new refresh token is returned - Hold it in process memory - On restart → back to step 1, reading the old refresh token If the old token is still valid, it works. But if it’s invalidated, authentication fails. Worse, the failure doesn’t appear until the process restarts, making the root cause hard to diagnose. The correct approach is to persist the new refresh token every time it’s updated. This issue appears in other services too, so it’s worth confirming whether you’re assuming “refresh tokens are immutable.” Plan Ahead for Where Sensitive Data Resides A quick operational note. Avoid pasting client secrets or refresh tokens into chats or tickets. We used a file-based approach, writing to a secure location and deleting it afterward. touch ~/.twitch-cred && chmod 600 ~/.twitch-cred # Write values # After use shred -u ~/.twitch-cred In a previous project, a stream key accidentally ended up in logs, forcing us to reset it. Plan ahead for where secrets end up. “ffmpeg Is Running” ≠ “Stream Is Live” Once authentication is sorted, the next step is sending video to Twitch. Here we hit a situation where everything looked normal on the sender side, yet the channel never went live. - ffmpeg was running and sending frames continuously - No errors in stderr - RTMP connection was maintained (no disconnections or reconnections) - Yet the channel remained offline Where to even start debugging? The root cause boiled down to two issues: 1. Twitch’s RTMP ingest accepts invalid stream keys and silently discards the data. From the sender’s side, success is indistinguishable. 2. If the stream itself is malformed, Twitch won’t mark the channel as live. Even the dashboard’s Stream Inspector shows nothing. A Persistent Connection Doesn’t Mean the Key Is Valid A common misconception with RTMP is that “if the connection is established and maintained, the key is valid.” That’s not true. The discrepancy between what we saw and Twitch’s state was: | What We Saw | State | |---|---| | ffmpeg logs | Normal. Frames being sent | | TCP connection | Established & maintained | | Twitch channel | Still offline | In our case, the cause was account-side settings (2FA and stream key status). Once fixed, running the exact same command made the channel go live. We hadn’t changed anything on the sender side. The operational rule we derived is simple: Determine streaming success not by sender-side logs, but by receiver-side state. “Malformed Streams” Also Won’t Go Live Another issue we encountered. At the time, we were generating video via CPU rendering. When we checked the recording, only 6 seconds of video were saved out of 90 seconds (due to an audio track exhaustion bug). Sending this timeline-dropped stream to Twitch resulted in a connection being established, but the channel never going live. The Stream Inspector showed no information. From Twitch’s perspective, it received a stream with timestamps far behind real time, making it impossible to treat as a valid broadcast. Again, from the sender’s side, it “looked fine.” Just because ffmpeg is running doesn’t mean a valid stream is being sent. How We Built External Monitoring We incorporated a mechanism to fetch Twitch’s state externally as part of our validation process. We used a public endpoint that returns the channel’s uptime. curl -s "https://decapi.me/twitch/uptime/ " # Live: "49 seconds" # Offline: " is offline" This allowed us to mechanically determine whether the channel was actually live. We changed our acceptance criteria for stream tests from sender-side logs to this output. However, there’s a trap. Third-party APIs like this often cache responses. If you check immediately after starting a stream, you might still get offline , leading to a false negative and wasted debugging effort. # Single checks are unreliable. Poll instead. for i in $(seq 1 20); do curl -s "https://decapi.me/twitch/uptime/ " echo sleep 15 done The same applies to stopping. After stopping the stream, offline isn’t immediate. For both start and stop, poll until the state stabilizes. “Sent” ≠ “Delivered” - Separate Checks This isn’t specific to Twitch. The same pattern appears in many places: - Email delivery: SMTP returns 250, but the message may not reach the inbox - Webhooks: 200 OK doesn’t guarantee the receiver processed it successfully - Metrics submission: The agent may send data, but it might not appear in the dashboard In all cases, there’s a gap between “sender-side success” and “receiver-side state.” Relying only on one side leads to silently broken states. After this incident, our streaming validation checklist always includes a line: “External confirmation of receiver state.” Silence is not a synonym for success. Three Ways to Crash, One of Which Keeps Running Silently Once streaming works, the next challenge is surviving unattended operation. How the system behaves when it crash