HydraExperienceLibrary

Experience lifecycle manager handling the full journey from draft through live deployment to retirement.

Infrastructure

Operations

Check status

ssh root@46.225.120.6 systemctl status hydraexperiencelibrary

Or via the health endpoint:

curl -s https://hydraexperiencelibrary.experiencenet.com/api/v1/health

View logs

ssh root@46.225.120.6 journalctl -u hydraexperiencelibrary -f

Show last 100 lines:

ssh root@46.225.120.6 journalctl -u hydraexperiencelibrary -n 100 --no-pager

Restart

ssh root@46.225.120.6 systemctl restart hydraexperiencelibrary

Update

Updates happen automatically. The service polls releases.experiencenet.com for new versions and applies them. To check the current version:

ssh root@46.225.120.6 hydraexperiencelibrary version

Never manually deploy -- always use the release pipeline (tag + push to trigger CI, release server distributes, service auto-updates).

Create experience (API)

Creates a new experience in draft state:

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Experience Name",
    "organization_id": "<organization-id>",
    "description": "Description of the experience"
  }'

List experiences (API)

curl -s https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences \
  -H "Authorization: Bearer <token>" | jq .

Promote experience (API)

Move an experience to the next lifecycle state (e.g., draft to staging, staging to live):

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/<id>/promote \
  -H "Authorization: Bearer <token>"

Pause experience (API)

Pause a live experience (can be resumed later):

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/<id>/pause \
  -H "Authorization: Bearer <token>"

Rollback experience (API)

Roll back an experience to a previous build version:

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/<id>/rollback \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "target_version": "<version>"
  }'

Retire experience (API)

Permanently retire an experience (terminal state):

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/<id>/retire \
  -H "Authorization: Bearer <token>"

List live experiences (API)

Returns all experiences with status live, a production build, and a non-empty build URL. Used by hydrabody to discover which experience builds to download:

curl -s https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/live \
  -H "Authorization: Bearer <token>" | jq .

Response:

[
  {"name": "rupelmonde-castle-viewer", "build_number": 1, "build_url": "https://releases.experiencenet.com/builds/...", "exe_path": "C:\\experiences\\rupelmonde-castle-viewer\\Rupelmonde.exe"}
]

Start stream (API)

Returns a URL to hydraheadwebstream for browser streaming of the given experience:

curl -X POST https://hydraexperiencelibrary.experiencenet.com/api/v1/experiences/<name>/stream \
  -H "Authorization: Bearer <token>"

Response:

{
  "stream_url": "https://hydraheadwebstream.experiencenet.com/?experience=<name>"
}

Requires hydraheadwebstream_url to be configured.

Key Concepts

• Lifecycle states: experiences progress through draft -> staging -> live -> paused -> retired. Promotion moves forward, rollback reverts to a previous build version. Paused experiences can be resumed to live. Retired is a terminal state.
• Build integration via SSE: the service listens for SSE events from HydraRelease. When a new build is released, it automatically becomes available for experiences to pick up. Agencies submit Unreal Engine builds via Perforce, which flow through HydraPerforce -> HydraRelease -> HydraExperienceLibrary.
• YAML store: index file plus per-entity YAML files in /root/.hydraexperiencelibrary/. Each experience file tracks its current state, build version history, and metadata. No database required.
• REST API with bearer token auth: all mutating endpoints require a valid bearer token issued by hydraauth.
• Browser streaming via hydraheadwebstream: the hydraheadwebstream_url config field enables stream links. Stream buttons redirect to hydraheadwebstream, which manages its own streaming sessions (moonlight-web-stream instances, Sunshine pairing, WebRTC).
• SSE events via hydramonitor: experience lifecycle changes are broadcast as server-sent events, consumed by HydraPipeline and other services.
• Auto-updates from releases.experiencenet.com: the service checks for new releases and updates itself without manual intervention.

Troubleshooting

YAML corruption

If the YAML store becomes corrupted (partial writes, disk issues):

1. Stop the service: systemctl stop hydraexperiencelibrary
2. Check the data directory: ls -la /root/.hydraexperiencelibrary/
3. Look for malformed YAML files -- inspect the index and individual entity files
4. Restore from backup if needed, or manually fix the YAML syntax
5. Start the service: systemctl start hydraexperiencelibrary

Service won't start

1. Check logs for the error: journalctl -u hydraexperiencelibrary -n 50 --no-pager
2. Verify config exists and is valid: cat /root/.hydraexperiencelibrary/config.yaml
3. Check if the port is already in use: ss -tlnp | grep <port>
4. Verify the binary exists and is executable: ls -la $(which hydraexperiencelibrary)
5. Try running manually to see output: hydraexperiencelibrary serve

Auth token issues

1. Verify the token by calling hydraauth directly: curl -s https://hydraauth.experiencenet.com/api/v1/verify -H "Authorization: Bearer <token>"
2. Check that hydraauth is reachable from the experience library server: ssh root@46.225.120.6 curl -s https://hydraauth.experiencenet.com/api/v1/health
3. If tokens are consistently rejected, check clock sync: ssh root@46.225.120.6 date -- token validation may fail if the server clock is skewed

Experience stuck in wrong state

1. Check the experience YAML file in /root/.hydraexperiencelibrary/ for the current state
2. Review logs for failed state transitions: journalctl -u hydraexperiencelibrary --since "1 hour ago" | grep <experience-id>
3. If the state is inconsistent, stop the service, manually correct the YAML file, and restart

Not picking up new builds from HydraRelease

1. Check that the SSE connection to HydraRelease is active in the logs
2. Verify HydraRelease is healthy: curl -s https://hydrarelease.experiencenet.com/api/v1/health
3. Check for network issues between the servers: ssh root@46.225.120.6 curl -s https://hydrarelease.experiencenet.com/api/v1/health
4. Restart the service to re-establish the SSE connection if needed