Discussions

Discussions

Ask a Question
Back to all

WebSocket Connection Failures on Railway.app Deployment (Pipecat Integration)

4 months ago by Luke P

Title:

WebSocket Connection Failures on Railway.app Deployment (Pipecat Integration)

Category:

Streaming API / Troubleshooting

Body:

Issue Summary

I'm experiencing persistent WebSocket disconnections (websocket is not connected anymore) when using HeyGen's Streaming API from a Railway.app deployment. The exact same code works perfectly on my local machine.

Environment Details

  • Deployment Platform: Railway.app (cloud hosting)
  • Framework: Pipecat (Python) with HeyGen integration
  • Programming Language: Python 3.11
  • Avatar: Using a custom avatar (tested with multiple different avatars - same issue)
  • Quality Setting: Medium (also tested with Low - same issue)

Configuration

session_request = NewSessionRequest(
    avatar_id="[CUSTOM_AVATAR_ID]",
    quality="medium",
    disable_idle_timeout=True,  # As recommended by support
)

heygen = HeyGenVideoService(
    api_key=os.getenv("HEYGEN_API_KEY"),
    session=session,
    session_request=session_request,
)

Symptoms

Local Development (macOS):

  • ✅ HeyGen initializes successfully
  • ✅ WebSocket connection stable
  • ✅ Video streaming works perfectly
  • ✅ Avatar responds to audio input
  • ✅ No disconnections

Railway.app Production:

  • ✅ HeyGen initializes (session created)
  • ✅ Bot connects to Daily.co room
  • ✅ OpenAI Realtime API works
  • ❌ WebSocket immediately disconnects
  • ❌ Constant websocket is not connected anymore errors
  • ❌ No video output

Error Logs from Production

The bot subprocess shows hundreds of these errors:

2025-11-18 10:01:09.030 | DEBUG | pipecat.services.heygen.client:_ws_send:263 - <pipecat.services.heygen.client.HeyGenClient object at 0x7fbd6f3eaba0> websocket is not connected anymore.
2025-11-18 10:01:09.030 | DEBUG | pipecat.services.heygen.client:_ws_send:263 - <pipecat.services.heygen.client.HeyGenClient object at 0x7fbd6f3eaba0> websocket is not connected anymore.
[... repeated hundreds of times ...]

The WebSocket disconnects immediately after initialization and never recovers.

What I've Already Tried

  1. ✅ Verified all environment variables - API keys are correctly set on Railway
  2. ✅ Added disable_idle_timeout=True - As recommended by support
  3. ✅ Implemented retry logic - 3 attempts with exponential backoff
  4. ✅ Tested multiple avatars - Same issue with different avatar IDs
  5. ✅ Lowered quality setting - Tried low quality, same issue
  6. ✅ Checked other services - OpenAI Realtime API and Daily.co work fine on Railway
  7. ✅ Confirmed HeyGen API accessible - Can successfully call list sessions endpoint from Railway

Network Details

What works on Railway:

  • OpenAI WebSocket connections (Realtime API)
  • Daily.co WebRTC connections
  • Standard HTTPS API calls to HeyGen (e.g., v1/streaming.list)

What doesn't work:

  • HeyGen WebSocket streaming connection

This suggests Railway's network can handle WebSockets in general, but something specific about HeyGen's WebSocket implementation or servers is incompatible.

Questions

  1. Are Railway.app IP addresses being throttled or blocked by HeyGen's WebSocket servers?

  2. Are there specific WebSocket connection parameters or headers required for cloud deployments?

  3. Does HeyGen's streaming API have known compatibility issues with certain cloud providers?

  4. Are there regional differences in HeyGen's WebSocket server stability? (Railway may use different data centers than my local ISP)

  5. Should I implement a WebSocket proxy or use a different connection strategy for production deployments?

  6. Are there any session states I should check for? (You mentioned checking for sessions stuck in "connecting," "connected," or "closing" states - how do I query this?)

Additional Context

  • Repository: GitHub (private repository - can share code snippets if needed)
  • Similar reports: Found general HeyGen instability issues on GitHub but no Railway-specific solutions
  • Support recommendation: Post to developer forum for engineering team assistance
  • Note: Happy to provide API key privately to HeyGen engineers for debugging if needed

What Would Help

  • Guidance on Railway.app compatibility
  • WebSocket debugging steps specific to cloud deployments
  • Alternative configuration options for production environments
  • Information about HeyGen's WebSocket infrastructure and requirements

Thank you for your help! This is blocking our production deployment. Happy to provide additional logs, network traces, or test configurations as needed.

Log in to add a comment.