| 1 |
# Running Django's Full Test Suite Against SQL Server |
| 2 |
|
| 3 |
This guide covers running Django's comprehensive test suite (~6000+ tests) against SQL Server using mssql-django. This is what CI runs and takes 30-45 minutes. |
| 4 |
|
| 5 |
## Prerequisites |
| 6 |
|
| 7 |
### 1. SQL Server Database (Required) |
| 8 |
|
| 9 |
```bash |
| 10 |
# Start SQL Server 2022 in Docker |
| 11 |
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=MyPassword42" \ |
| 12 |
-p 1433:1433 --name sqlserver -d mcr.microsoft.com/mssql/server:2022-latest |
| 13 |
|
| 14 |
# Wait for SQL Server to fully initialize |
| 15 |
sleep 25 |
| 16 |
|
| 17 |
# Verify it's running |
| 18 |
docker logs sqlserver 2>&1 | tail -3 |
| 19 |
# Should show: "Recovery is complete" and "tempdb database has X data file(s)" |
| 20 |
``` |
| 21 |
|
| 22 |
### 2. ODBC Driver (Required) |
| 23 |
|
| 24 |
```bash |
| 25 |
# Ubuntu 24.04 - Install ODBC Driver 18 |
| 26 |
curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | sudo gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg |
| 27 |
echo "deb [arch=amd64,arm64,armhf signed-by=/usr/share/keyrings/microsoft-prod.gpg] https://packages.microsoft.com/ubuntu/24.04/prod noble main" | sudo tee /etc/apt/sources.list.d/mssql-release.list |
| 28 |
sudo apt-get update |
| 29 |
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql18 unixodbc-dev |
| 30 |
``` |
| 31 |
|
| 32 |
### 3. System Dependencies (Required for Django test dependencies) |
| 33 |
|
| 34 |
```bash |
| 35 |
# libmemcached is needed for pylibmc (Django test dependency) |
| 36 |
sudo apt-get install -y libmemcached-dev |
| 37 |
``` |
| 38 |
|
| 39 |
### 4. Python Dependencies |
| 40 |
|
| 41 |
```bash |
| 42 |
# Install mssql-django in development mode |
| 43 |
pip install -e . |
| 44 |
|
| 45 |
# Install coverage (used by test.sh) |
| 46 |
pip install coverage |
| 47 |
``` |
| 48 |
|
| 49 |
### 5. Update Settings for ODBC Driver 18 |
| 50 |
|
| 51 |
If using ODBC Driver 18 (Ubuntu 24.04), update `testapp/settings.py`: |
| 52 |
|
| 53 |
```python |
| 54 |
DATABASES = { |
| 55 |
"default": { |
| 56 |
"ENGINE": "mssql", |
| 57 |
"NAME": "default", |
| 58 |
"USER": "sa", |
| 59 |
"PASSWORD": "MyPassword42", |
| 60 |
"HOST": "localhost", |
| 61 |
"PORT": "1433", |
| 62 |
"OPTIONS": { |
| 63 |
"driver": "ODBC Driver 18 for SQL Server", |
| 64 |
"return_rows_bulk_insert": True, |
| 65 |
"extra_params": "TrustServerCertificate=yes" |
| 66 |
}, |
| 67 |
}, |
| 68 |
'other': { |
| 69 |
"ENGINE": "mssql", |
| 70 |
"NAME": "other", |
| 71 |
"USER": "sa", |
| 72 |
"PASSWORD": "MyPassword42", |
| 73 |
"HOST": "localhost", |
| 74 |
"PORT": "1433", |
| 75 |
"OPTIONS": { |
| 76 |
"driver": "ODBC Driver 18 for SQL Server", |
| 77 |
"return_rows_bulk_insert": True, |
| 78 |
"extra_params": "TrustServerCertificate=yes" |
| 79 |
}, |
| 80 |
}, |
| 81 |
} |
| 82 |
``` |
| 83 |
|
| 84 |
### 6. Django Repository |
| 85 |
|
| 86 |
The `test.sh` script expects a pre-existing Django clone in the `django/` directory. It uses `git fetch`, not `git clone`: |
| 87 |
|
| 88 |
```bash |
| 89 |
cd /workspaces/mssql-django |
| 90 |
git clone --depth=1 https://github.com/django/django.git django |
| 91 |
``` |
| 92 |
|
| 93 |
## Running the Full Test Suite |
| 94 |
|
| 95 |
### Using test.sh (Recommended) |
| 96 |
|
| 97 |
```bash |
| 98 |
cd /workspaces/mssql-django |
| 99 |
bash test.sh 2>&1 | tee test_output.log |
| 100 |
``` |
| 101 |
|
| 102 |
This will: |
| 103 |
1. Fetch tags and checkout the Django version matching your installed Django |
| 104 |
2. Install Django's test requirements |
| 105 |
3. Run ~100+ test modules against SQL Server |
| 106 |
4. Generate coverage report |
| 107 |
|
| 108 |
**Expected runtime:** 30-45 minutes |
| 109 |
|
| 110 |
### Manual Execution |
| 111 |
|
| 112 |
```bash |
| 113 |
cd /workspaces/mssql-django/django |
| 114 |
|
| 115 |
# Fetch Django tags and checkout matching version |
| 116 |
DJANGO_VERSION="$(python -m django --version)" |
| 117 |
git fetch --depth=1 origin +refs/tags/*:refs/tags/* |
| 118 |
git checkout $DJANGO_VERSION |
| 119 |
|
| 120 |
# Install Django test requirements |
| 121 |
pip install -r tests/requirements/py3.txt |
| 122 |
|
| 123 |
# Run tests |
| 124 |
coverage run tests/runtests.py --settings=testapp.settings --noinput \ |
| 125 |
aggregation \ |
| 126 |
annotations \ |
| 127 |
basic \ |
| 128 |
composite_pk \ |
| 129 |
queries \ |
| 130 |
# ... add more test modules as needed |
| 131 |
``` |
| 132 |
|
| 133 |
### Run Specific Test Modules |
| 134 |
|
| 135 |
```bash |
| 136 |
cd /workspaces/mssql-django/django |
| 137 |
python tests/runtests.py --settings=testapp.settings --noinput composite_pk |
| 138 |
``` |
| 139 |
|
| 140 |
### Run Specific Test |
| 141 |
|
| 142 |
```bash |
| 143 |
cd /workspaces/mssql-django/django |
| 144 |
python tests/runtests.py --settings=testapp.settings --noinput \ |
| 145 |
composite_pk.test_filter.CompositePKFilterTests.test_explicit_subquery |
| 146 |
``` |
| 147 |
|
| 148 |
## Understanding Test Output |
| 149 |
|
| 150 |
Test result symbols: |
| 151 |
- `.` = passed |
| 152 |
- `x` = expected failure (test is known to fail on SQL Server) |
| 153 |
- `s` = skipped (test excluded in settings.py) |
| 154 |
- `E` = error |
| 155 |
- `F` = failure |
| 156 |
|
| 157 |
## Test Exclusions |
| 158 |
|
| 159 |
Tests that are known to fail on SQL Server are excluded in `testapp/settings.py`: |
| 160 |
|
| 161 |
```python |
| 162 |
EXCLUDED_TESTS = [ |
| 163 |
# Tuple lookups - SQL Server doesn't support (col1, col2) IN syntax |
| 164 |
'composite_pk.test_filter.CompositePKFilterTests.test_explicit_subquery', |
| 165 |
'foreign_object.test_tuple_lookups.TupleLookupsTests.test_exact', |
| 166 |
# ... many more |
| 167 |
] |
| 168 |
``` |
| 169 |
|
| 170 |
## Expected Results |
| 171 |
|
| 172 |
From CI (Django 5.2, Python 3.13): |
| 173 |
- **Total tests:** ~6200 |
| 174 |
- **Passed:** ~5400+ |
| 175 |
- **Skipped:** ~500+ |
| 176 |
- **Expected failures:** ~200+ |
| 177 |
- **Errors:** 0 (if code is working correctly) |
| 178 |
|
| 179 |
## Cleanup |
| 180 |
|
| 181 |
```bash |
| 182 |
# Stop SQL Server |
| 183 |
docker stop sqlserver && docker rm sqlserver |
| 184 |
|
| 185 |
# Restore settings.py |
| 186 |
git checkout testapp/settings.py |
| 187 |
|
| 188 |
# Remove Django clone (optional) |
| 189 |
rm -rf django/ |
| 190 |
``` |
| 191 |
|
| 192 |
## Troubleshooting |
| 193 |
|
| 194 |
### "coverage: command not found" |
| 195 |
|
| 196 |
```bash |
| 197 |
pip install coverage |
| 198 |
``` |
| 199 |
|
| 200 |
### "pylibmc build failed" |
| 201 |
|
| 202 |
```bash |
| 203 |
sudo apt-get install -y libmemcached-dev |
| 204 |
``` |
| 205 |
|
| 206 |
### Tests hang or timeout |
| 207 |
|
| 208 |
Check SQL Server is running: |
| 209 |
```bash |
| 210 |
docker ps | grep sqlserver |
| 211 |
docker logs sqlserver | tail -10 |
| 212 |
``` |
| 213 |
|
| 214 |
### "An expression of non-boolean type specified in a context where a condition is expected" |
| 215 |
|
| 216 |
This is the tuple lookup limitation. The test should be in `EXCLUDED_TESTS` in settings.py. |
| 217 |
|
| 218 |
### "A column has been specified more than once in the order by list" |
| 219 |
|
| 220 |
This is the ORDER BY duplicate column issue. If you see this, the deduplication fix in `mssql/compiler.py` may need adjustment. |
| 221 |
|
| 222 |
## CI Configuration |
| 223 |
|
| 224 |
The CI uses `tox` with matrix testing. See: |
| 225 |
- `azure-pipelines.yml` - CI configuration |
| 226 |
- `tox.ini` - Test matrix (Python versions × Django versions) |
| 227 |
- `test.sh` - Test script that CI runs |
| 228 |
|
| 229 |
## Test Modules Covered |
| 230 |
|
| 231 |
The full list of Django test modules run by `test.sh`: |
| 232 |
|
| 233 |
- aggregation, aggregation_regress |
| 234 |
- annotations, backends, basic |
| 235 |
- bulk_create, composite_pk, constraints |
| 236 |
- custom_columns, custom_lookups, custom_managers |
| 237 |
- custom_methods, custom_pk, datatypes |
| 238 |
- dates, datetimes, db_functions |
| 239 |
- defer, delete, expressions |
| 240 |
- fixtures, foreign_object, get_or_create |
| 241 |
- indexes, inspectdb, introspection |
| 242 |
- lookup, m2m_*, many_to_one, many_to_many |
| 243 |
- migrations, migrations2, model_fields |
| 244 |
- ordering, pagination, prefetch_related |
| 245 |
- queries, raw_query, schema |
| 246 |
- select_for_update, select_related |
| 247 |
- serializers, timezones, transactions |
| 248 |
- update, update_only_fields |
| 249 |
- ... and more (see test.sh for complete list) |