| 1 |
# Matrix E2E Testing |
| 2 |
|
| 3 |
This directory contains black-box end-to-end (E2E) tests for Shoutrrr's Matrix service. |
| 4 |
These tests validate functionality by connecting to a real Matrix Synapse server and sending actual messages. |
| 5 |
|
| 6 |
## Overview |
| 7 |
|
| 8 |
These are **black-box E2E tests** that verify the Matrix service works correctly with a live Matrix server. |
| 9 |
The tests make real HTTP connections to the Matrix server's Client-Server API. |
| 10 |
|
| 11 |
## Test Characteristics |
| 12 |
|
| 13 |
- **Package**: `e2e_test` |
| 14 |
- **Framework**: [Ginkgo](https://github.com/onsi/ginkgo) + [Gomega](https://github.com/onsi/gomega) |
| 15 |
- **Location**: `testing/e2e/matrix` |
| 16 |
- **External Calls**: Yes (requires running Matrix server) |
| 17 |
- **Requires**: Docker and Docker Compose for local testing |
| 18 |
|
| 19 |
## Test Coverage |
| 20 |
|
| 21 |
### Service Initialization |
| 22 |
|
| 23 |
- Initialization with valid credentials from environment variables |
| 24 |
- Error handling for invalid credentials |
| 25 |
- Error handling for missing host |
| 26 |
- Error handling for missing credentials |
| 27 |
|
| 28 |
### Send Messages |
| 29 |
|
| 30 |
- Sending basic text messages to Matrix rooms |
| 31 |
- Sending messages with custom title parameter |
| 32 |
- Error handling when client is not initialized |
| 33 |
|
| 34 |
### Service ID |
| 35 |
|
| 36 |
- Verifying correct service ID is returned |
| 37 |
|
| 38 |
### Configuration |
| 39 |
|
| 40 |
- Room alias parsing and handling |
| 41 |
|
| 42 |
## Running the Test Environment |
| 43 |
|
| 44 |
## Quick Setup |
| 45 |
|
| 46 |
Use the provided setup script to automate all setup steps: |
| 47 |
|
| 48 |
```bash |
| 49 |
# Run full setup (generate config, start server, create user, create room) |
| 50 |
./testing/e2e/matrix/setup.sh |
| 51 |
|
| 52 |
# Or with explicit command |
| 53 |
./testing/e2e/matrix/setup.sh setup-all |
| 54 |
``` |
| 55 |
|
| 56 |
Individual steps can also be run separately: |
| 57 |
|
| 58 |
```bash |
| 59 |
./testing/e2e/matrix/setup.sh generate-config # Generate Docker config |
| 60 |
./testing/e2e/matrix/setup.sh start-server # Start the server |
| 61 |
./testing/e2e/matrix/setup.sh create-user # Create admin user |
| 62 |
./testing/e2e/matrix/setup.sh create-room # Create test room |
| 63 |
``` |
| 64 |
|
| 65 |
The script automatically finds the `.env` file in its directory. |
| 66 |
|
| 67 |
### Prerequisites |
| 68 |
|
| 69 |
1. **Generate the configuration** |
| 70 |
|
| 71 |
Run the following to generate a valid config file: |
| 72 |
|
| 73 |
```bash |
| 74 |
docker run -it --rm \ |
| 75 |
-v ./data:/data \ |
| 76 |
--env-file element.env \ |
| 77 |
matrixdotorg/synapse:latest \ |
| 78 |
generate |
| 79 |
``` |
| 80 |
|
| 81 |
2. **Start the Matrix Server** |
| 82 |
|
| 83 |
Use the provided `docker-compose.yaml` to start a local Matrix Synapse server: |
| 84 |
|
| 85 |
```bash |
| 86 |
cd testing/e2e/matrix |
| 87 |
docker compose up -d |
| 88 |
``` |
| 89 |
|
| 90 |
> [!Note] |
| 91 |
> A fresh deployment will take a few minutes to startup. |
| 92 |
> Use `docker logs synapse -f` to watch the logs. |
| 93 |
|
| 94 |
3. **Create a Test User** |
| 95 |
|
| 96 |
After starting the server, register a test user: |
| 97 |
|
| 98 |
```bash |
| 99 |
docker exec synapse \ |
| 100 |
register_new_matrix_user \ |
| 101 |
-u admin -p admin \ |
| 102 |
-a http://localhost:8008 \ |
| 103 |
-c /data/homeserver.yaml |
| 104 |
``` |
| 105 |
|
| 106 |
4. **Create a Test Room** |
| 107 |
|
| 108 |
After creating the admin user, create a test room with the alias `#test:localhost`: |
| 109 |
|
| 110 |
> [!Tip] |
| 111 |
> An automated script is available to create the test room: |
| 112 |
> |
| 113 |
> ```bash |
| 114 |
> bash ./testing/e2e/matrix/create-room.sh |
| 115 |
> ``` |
| 116 |
> |
| 117 |
> The script automatically finds the `.env` file in its directory. |
| 118 |
> Alternatively, you can run it directly if the file is executable: |
| 119 |
> |
| 120 |
> ```bash |
| 121 |
> ./testing/e2e/matrix/create-room.sh |
| 122 |
> ``` |
| 123 |
> |
| 124 |
> This provides an easier alternative to the manual curl commands below. |
| 125 |
|
| 126 |
Alternatively, you can manually create the room using curl commands: |
| 127 |
|
| 128 |
First, login to get an access token: |
| 129 |
|
| 130 |
```bash |
| 131 |
# Login to get access token |
| 132 |
TOKEN=$(curl -s -X POST \ |
| 133 |
-H "Content-Type: application/json" \ |
| 134 |
-d '{"type": "m.login.password", "identifier": {"type": "m.id.user", "user": "admin"}, "password": "admin"}' \ |
| 135 |
http://localhost:8008/_matrix/client/v3/login | jq -r '.access_token') |
| 136 |
|
| 137 |
# Create a room with the alias #test:localhost |
| 138 |
ROOM_ID=$(curl -s -X POST \ |
| 139 |
-H "Authorization: Bearer $TOKEN" \ |
| 140 |
-H "Content-Type: application/json" \ |
| 141 |
-d '{"name": "Test Room", "room_alias_name": "test"}' \ |
| 142 |
http://localhost:8008/_matrix/client/v3/createRoom | jq -r '.room_id') |
| 143 |
|
| 144 |
echo "Created room: $ROOM_ID with alias #test:localhost" |
| 145 |
``` |
| 146 |
|
| 147 |
Or alternatively using a Matrix client like Element or any other Matrix client: |
| 148 |
|
| 149 |
1. Open Element (or any Matrix client) |
| 150 |
2. Connect to your local server at `http://localhost:8008` |
| 151 |
3. Login with the admin user (`admin` / `admin`) |
| 152 |
4. Create a new room with the alias `#test:localhost` |
| 153 |
|
| 154 |
> [!Important] |
| 155 |
> The test room `#test:localhost` must exist for the E2E tests to pass. |
| 156 |
> The tests expect this room to be available on the local Synapse server. |
| 157 |
|
| 158 |
### Environment Variables |
| 159 |
|
| 160 |
Configure the tests using environment variables: |
| 161 |
|
| 162 |
| Variable | Description | Default | |
| 163 |
|-------------------------------|--------------------------------|-------------------| |
| 164 |
| `SHOUTRRR_MATRIX_URL` | Full Matrix service URL | - | |
| 165 |
| `SHOUTRRR_MATRIX_HOST` | Matrix server host | `localhost:8008` | |
| 166 |
| `SHOUTRRR_MATRIX_USER` | Username for authentication | - | |
| 167 |
| `SHOUTRRR_MATRIX_PASSWORD` | Password or access token | - | |
| 168 |
| `SHOUTRRR_MATRIX_ROOM` | Room alias to send messages to | `#test:localhost` | |
| 169 |
| `SHOUTRRR_MATRIX_DISABLE_TLS` | Disable TLS verification | `false` | |
| 170 |
|
| 171 |
### Running the Tests |
| 172 |
|
| 173 |
Run all E2E tests: |
| 174 |
|
| 175 |
```bash |
| 176 |
go test -v ./testing/e2e/matrix/... |
| 177 |
``` |
| 178 |
|
| 179 |
Run specific test suites: |
| 180 |
|
| 181 |
```bash |
| 182 |
# Run service initialization tests |
| 183 |
go test ./testing/e2e/matrix/ -v -ginkgo.focus "Service Initialization" |
| 184 |
|
| 185 |
# Run send message tests |
| 186 |
go test ./testing/e2e/matrix/ -v -ginkgo.focus "Send Messages" |
| 187 |
``` |
| 188 |
|
| 189 |
### Using .env File |
| 190 |
|
| 191 |
You can also create a `.env` file in the test directory: |
| 192 |
|
| 193 |
```bash |
| 194 |
# .env file |
| 195 |
SHOUTRRR_MATRIX_HOST=localhost:8008 |
| 196 |
SHOUTRRR_MATRIX_USER=admin |
| 197 |
SHOUTRRR_MATRIX_PASSWORD=admin |
| 198 |
SHOUTRRR_MATRIX_ROOM=#test:localhost |
| 199 |
SHOUTRRR_MATRIX_DISABLE_TLS=true |
| 200 |
``` |
| 201 |
|
| 202 |
The test suite automatically loads environment variables from `.env` if present. |
| 203 |
|
| 204 |
## Test Structure |
| 205 |
|
| 206 |
```bash |
| 207 |
testing/e2e/matrix/ |
| 208 |
├── matrix_suite_test.go # Ginkgo test suite setup and helpers |
| 209 |
├── service_e2e_test.go # E2E test cases |
| 210 |
├── docker-compose.yaml # Local Matrix Synapse server |
| 211 |
└── README.md # This documentation |
| 212 |
``` |
| 213 |
|
| 214 |
## Test Suite Helpers |
| 215 |
|
| 216 |
### matrixServerURL |
| 217 |
|
| 218 |
Returns the Matrix server URL from environment or builds it from components: |
| 219 |
|
| 220 |
```go |
| 221 |
func matrixServerURL() string |
| 222 |
``` |
| 223 |
|
| 224 |
### matrixRoom |
| 225 |
|
| 226 |
Returns the Matrix room from environment or default: |
| 227 |
|
| 228 |
```go |
| 229 |
func matrixRoom() string |
| 230 |
``` |
| 231 |
|
| 232 |
### buildServiceURL |
| 233 |
|
| 234 |
Builds a complete Matrix service URL with all parameters: |
| 235 |
|
| 236 |
```go |
| 237 |
func buildServiceURL() string |
| 238 |
``` |
| 239 |
|
| 240 |
### loadEnvFile |
| 241 |
|
| 242 |
Loads environment variables from a `.env` file: |
| 243 |
|
| 244 |
```go |
| 245 |
func loadEnvFile(filename string) |
| 246 |
``` |
| 247 |
|
| 248 |
## Test Patterns |
| 249 |
|
| 250 |
Tests use Ginkgo'sDescribe/It syntax: |
| 251 |
|
| 252 |
```go |
| 253 |
var _ = ginkgo.Describe("Matrix Service E2E Tests", func() { |
| 254 |
ginkgo.Describe("Service Initialization", func() { |
| 255 |
ginkgo.It("should initialize with valid credentials", func() { |
| 256 |
// Test logic |
| 257 |
}) |
| 258 |
}) |
| 259 |
}) |
| 260 |
``` |
| 261 |
|
| 262 |
## Skipping Tests |
| 263 |
|
| 264 |
Tests automatically skip when environment variables are not configured: |
| 265 |
|
| 266 |
```go |
| 267 |
if serviceURL == "" { |
| 268 |
ginkgo.Skip("Matrix server not configured, skipping test...") |
| 269 |
} |
| 270 |
``` |
| 271 |
|
| 272 |
## Docker Compose Configuration |
| 273 |
|
| 274 |
The provided `docker-compose.yaml` starts a Matrix Synapse server with: |
| 275 |
|
| 276 |
- **Image**: `matrixdotorg/synapse:latest` |
| 277 |
- **Ports**: |
| 278 |
- `8008` - Client-Server API |
| 279 |
- `8448` - Federation API |
| 280 |
- **Server Name**: `localhost` |
| 281 |
- **Registration**: Enabled for testing |
| 282 |
|
| 283 |
## Cleanup |
| 284 |
|
| 285 |
To stop and remove the Docker container: |
| 286 |
|
| 287 |
```bash |
| 288 |
docker compose down |
| 289 |
``` |
| 290 |
|
| 291 |
To remove the data volume: |
| 292 |
|
| 293 |
```bash |
| 294 |
sudo rm -rf ./testing/e2e/matrix/data |
| 295 |
``` |
| 296 |
|
| 297 |
## References |
| 298 |
|
| 299 |
- [Element Synapse GitHub](https://github.com/element-hq/synapse) |
| 300 |
- [Element Synapse Documentation](https://element-hq.github.io/synapse/latest/) |
| 301 |
- [Matrix Client-Server API](https://matrix.org/docs/spec/client_server/latest) |
| 302 |
- [Synapse Docker Image](https://hub.docker.com/r/matrixdotorg/synapse) |
| 303 |
- [Synapse Docker Documentation](https://github.com/element-hq/synapse/blob/develop/docker/README.md) |
| 304 |
|
| 305 |
## Related Files |
| 306 |
|
| 307 |
- Matrix Service Implementation: `pkg/services/chat/matrix/service.go` |
| 308 |
- Matrix Configuration: `pkg/services/chat/matrix/config.go` |
| 309 |
- Integration Tests: `testing/integration/matrix/` |