Skip to content

Backend Testing

Overview

The backend testing suite validates the API endpoints that serve data to the Digital Landscape application. These tests ensure that the backend correctly processes requests, applies filters, and returns properly structured data.

Test Implementation

The backend tests are implemented in the testing/backend/ directory using the pytest framework and the requests library to make HTTP calls to the API endpoints. The tests are organized into three main files:

  • test_main.py - Tests for core API endpoints
  • test_admin.py - Tests for admin API endpoints
  • test_review.py - Tests for review API endpoints

Base Configuration

All tests use a common base URL configuration:

BASE_URL = "http://localhost:5001"

Running Tests

The testing framework provides several commands for running tests:

# Run all tests
make test

# Run only main API tests
make test-main

# Run only admin API tests
make test-admin

# Run only review API tests
make test-review

Health Check Tests

The health check endpoint test verifies that the server is operational and returns basic health metrics:

Test the health check endpoint functionality.

This test verifies that the health check endpoint is operational and returns the expected health status information about the server. It checks for the presence of essential health metrics and status indicators.

Endpoint

GET /api/health

Expects
  • 200 status code
  • JSON response containing:
    • "healthy" status indicator
    • Current timestamp
    • Server uptime in seconds
    • Memory usage statistics
    • Process ID
Source code in testing/backend/test_main.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def test_health_check():
    """Test the health check endpoint functionality.

    This test verifies that the health check endpoint is operational and returns
    the expected health status information about the server. It checks for the
    presence of essential health metrics and status indicators.

    Endpoint:
        GET /api/health

    Expects:
        - 200 status code
        - JSON response containing:
            - "healthy" status indicator
            - Current timestamp
            - Server uptime in seconds
            - Memory usage statistics
            - Process ID
    """
    response = requests.get(f"{BASE_URL}/api/health", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert data["status"] == "healthy"
    assert "timestamp" in data
    assert "uptime" in data
    assert "memory" in data
    assert "pid" in data

Project Data Tests

The CSV endpoint test verifies that project data is correctly retrieved and formatted:

Test the CSV data endpoint functionality.

This test verifies that the CSV endpoint correctly returns parsed CSV data from the S3 bucket. It checks that the data is properly formatted and contains the expected structure.

Endpoint

GET /api/csv

Expects
  • 200 status code
  • JSON array response
  • Non-empty data entries
  • Each entry should be a dictionary with multiple fields
  • No empty or malformed entries
Source code in testing/backend/test_main.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
def test_csv_endpoint():
    """Test the CSV data endpoint functionality.

    This test verifies that the CSV endpoint correctly returns parsed CSV data
    from the S3 bucket. It checks that the data is properly formatted and
    contains the expected structure.

    Endpoint:
        GET /api/csv

    Expects:
        - 200 status code
        - JSON array response
        - Non-empty data entries
        - Each entry should be a dictionary with multiple fields
        - No empty or malformed entries
    """
    response = requests.get(f"{BASE_URL}/api/csv", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert isinstance(data, list)
    if len(data) > 0:
        first_item = data[0]
        assert isinstance(first_item, dict)
        assert len(first_item.keys()) > 1  # Verify it's not empty

Tech Radar Data Tests

The Tech Radar JSON endpoint test verifies that the radar configuration data is correctly retrieved:

Test the tech radar JSON endpoint functionality.

This test verifies that the tech radar endpoint correctly returns the radar configuration data from the S3 bucket. The data defines the structure and content of the technology radar visualization.

Endpoint

GET /api/tech-radar/json

Expects
  • 200 status code
  • JSON object response
  • Non-empty configuration data
  • Multiple configuration keys present
Source code in testing/backend/test_main.py 
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
def test_tech_radar_json_endpoint():
    """Test the tech radar JSON endpoint functionality.

    This test verifies that the tech radar endpoint correctly returns the
    radar configuration data from the S3 bucket. The data defines the structure
    and content of the technology radar visualization.

    Endpoint:
        GET /api/tech-radar/json

    Expects:
        - 200 status code
        - JSON object response
        - Non-empty configuration data
        - Multiple configuration keys present
    """
    response = requests.get(f"{BASE_URL}/api/tech-radar/json", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert isinstance(data, dict)
    assert len(data.keys()) > 1  # Verify it's not empty

Repository Statistics Tests

Basic Statistics

Tests the default behavior with no filters:

Test the JSON endpoint without query parameters.

This test verifies the default behavior of the JSON endpoint when no filters are applied. It checks that the endpoint returns complete repository statistics and metadata.

Endpoint

GET /api/json

Expects
  • 200 status code
  • JSON response containing:
    • Repository statistics
    • Language usage statistics
    • Metadata information
  • Complete stats structure with:
    • Total repository count
    • Private repository count
    • Public repository count
    • Internal repository count
Source code in testing/backend/test_main.py 
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
def test_json_endpoint_no_params():
    """Test the JSON endpoint without query parameters.

    This test verifies the default behavior of the JSON endpoint when no
    filters are applied. It checks that the endpoint returns complete
    repository statistics and metadata.

    Endpoint:
        GET /api/json

    Expects:
        - 200 status code
        - JSON response containing:
            - Repository statistics
            - Language usage statistics
            - Metadata information
        - Complete stats structure with:
            - Total repository count
            - Private repository count
            - Public repository count
            - Internal repository count
    """
    response = requests.get(f"{BASE_URL}/api/json", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert "stats" in data
    assert "language_statistics" in data
    assert "metadata" in data

    stats = data["stats"]
    assert "total_repos" in stats
    assert "total_private_repos" in stats
    assert "total_public_repos" in stats
    assert "total_internal_repos" in stats

Date Filtering

Tests filtering repositories by a specific date:

Test the JSON endpoint with datetime filtering.

This test verifies that the endpoint correctly filters repository data based on a specified datetime parameter. It checks repositories modified within the last 7 days.

Parameters:

Name Type Description Default
datetime str

ISO formatted datetime string for filtering

 required
Example

GET /api/json?datetime=2024-03-20T00:00:00Z

Expects
  • 200 status code
  • Filtered repository data
  • Metadata containing the applied datetime filter
Source code in testing/backend/test_main.py 
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
def test_json_endpoint_with_datetime():
    """Test the JSON endpoint with datetime filtering.

    This test verifies that the endpoint correctly filters repository data
    based on a specified datetime parameter. It checks repositories modified
    within the last 7 days.

    Parameters:
        datetime (str): ISO formatted datetime string for filtering

    Example:
        GET /api/json?datetime=2024-03-20T00:00:00Z

    Expects:
        - 200 status code
        - Filtered repository data
        - Metadata containing the applied datetime filter
    """
    seven_days_ago = (datetime.now() - timedelta(days=7)).isoformat()
    response = requests.get(f"{BASE_URL}/api/json",
                            params={"datetime": seven_days_ago}, timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert data["metadata"]["filter_date"] == seven_days_ago

Archived Status Filtering

Tests filtering repositories by archived status:

Test the JSON endpoint with archived status filtering.

This test verifies that the endpoint correctly filters repositories based on their archived status. It tests both archived and non-archived filtering options.

Parameters:

Name Type Description Default
archived str

"true" or "false" to filter archived status

 required
Example

GET /api/json?archived=false

Expects
  • 200 status code for both archived and non-archived queries
  • Filtered repository data based on archived status
Source code in testing/backend/test_main.py 
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
def test_json_endpoint_with_archived():
    """Test the JSON endpoint with archived status filtering.

    This test verifies that the endpoint correctly filters repositories
    based on their archived status. It tests both archived and non-archived
    filtering options.

    Parameters:
        archived (str): "true" or "false" to filter archived status

    Example:
        GET /api/json?archived=false

    Expects:
        - 200 status code for both archived and non-archived queries
        - Filtered repository data based on archived status
    """
    response = requests.get(f"{BASE_URL}/api/json",
                            params={"archived": "true"}, timeout=10)
    assert response.status_code == 200

    response = requests.get(f"{BASE_URL}/api/json",
                            params={"archived": "false"}, timeout=10)
    assert response.status_code == 200

Combined Filtering

Tests applying multiple filters simultaneously:

Test the JSON endpoint with multiple filter parameters.

This test verifies that the endpoint correctly handles multiple filter parameters simultaneously, including datetime and archived status filters. It ensures all filters are properly applied and reflected in the response.

Parameters:

Name Type Description Default
datetime str

ISO formatted datetime string for filtering

 required
archived str

"true" or "false" to filter archived status

 required
Example

GET /api/json?datetime=2024-03-20T00:00:00Z&archived=false

Expects
  • 200 status code
  • Repository data filtered by all parameters
  • Metadata reflecting the applied datetime filter
Source code in testing/backend/test_main.py 
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def test_json_endpoint_combined_params():
    """Test the JSON endpoint with multiple filter parameters.

    This test verifies that the endpoint correctly handles multiple filter
    parameters simultaneously, including datetime and archived status filters.
    It ensures all filters are properly applied and reflected in the response.

    Parameters:
        datetime (str): ISO formatted datetime string for filtering
        archived (str): "true" or "false" to filter archived status

    Example:
        GET /api/json?datetime=2024-03-20T00:00:00Z&archived=false

    Expects:
        - 200 status code
        - Repository data filtered by all parameters
        - Metadata reflecting the applied datetime filter
    """
    seven_days_ago = (datetime.now() - timedelta(days=7)).isoformat()
    params = {
        "datetime": seven_days_ago,
        "archived": "false"
    }
    response = requests.get(f"{BASE_URL}/api/json", params=params, timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert data["metadata"]["filter_date"] == seven_days_ago

Repository Project Tests

Error Handling

Tests the endpoint's response when required parameters are missing:

Test the repository project JSON endpoint error handling for missing parameters.

This test verifies that the endpoint correctly handles the case when no repositories are specified in the request parameters. It should return a 400 Bad Request status code with an appropriate error message.

Expects
  • 400 status code
  • JSON response with error message
  • Error message indicating no repositories specified
Source code in testing/backend/test_main.py 
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
def test_repository_project_json_no_params():
    """Test the repository project JSON endpoint error handling for missing parameters.

    This test verifies that the endpoint correctly handles the case when no
    repositories are specified in the request parameters. It should return
    a 400 Bad Request status code with an appropriate error message.

    Expects:
        - 400 status code
        - JSON response with error message
        - Error message indicating no repositories specified
    """
    response = requests.get(
        f"{BASE_URL}/api/repository/project/json", timeout=10)
    assert response.status_code == 400
    data = response.json()
    assert "error" in data
    assert data["error"] == "No repositories specified"

Single Repository

Tests retrieving data for a single repository:

Test the repository project JSON endpoint with a valid repository parameter.

This test verifies the endpoint's basic functionality when requesting data for a single repository. It checks the complete response structure including repository data, statistics, and metadata.

Parameters:

Name Type Description Default
repositories str

Name of the repository to query (e.g., "tech-radar")

 required
Expects
  • 200 status code
  • JSON response with complete repository data
  • Valid statistics for the repository
  • Correct metadata including requested repository names
  • Language statistics if available
Source code in testing/backend/test_main.py 
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
def test_repository_project_json_with_repos():
    """Test the repository project JSON endpoint with a valid repository parameter.

    This test verifies the endpoint's basic functionality when requesting data
    for a single repository. It checks the complete response structure including
    repository data, statistics, and metadata.

    Parameters:
        repositories (str): Name of the repository to query (e.g., "tech-radar")

    Expects:
        - 200 status code
        - JSON response with complete repository data
        - Valid statistics for the repository
        - Correct metadata including requested repository names
        - Language statistics if available
    """
    response = requests.get(f"{BASE_URL}/api/repository/project/json",
                            params={"repositories": "tech-radar"}, timeout=10)
    assert response.status_code == 200
    data = response.json()

    # Verify response structure
    assert "repositories" in data
    assert "stats" in data
    assert "language_statistics" in data
    assert "metadata" in data

    # Verify stats structure
    stats = data["stats"]
    assert "total_repos" in stats
    assert "total_private_repos" in stats
    assert "total_public_repos" in stats
    assert "total_internal_repos" in stats

    # Verify metadata
    metadata = data["metadata"]
    assert "requested_repos" in metadata
    assert "found_repos" in metadata
    assert metadata["requested_repos"] == ["tech-radar"]

Multiple Repositories

Tests retrieving data for multiple repositories:

Test the repository project JSON endpoint with multiple repositories.

This test verifies that the endpoint correctly handles requests for multiple repositories in a single call. It checks that all requested repositories are processed and included in the response.

Parameters:

Name Type Description Default
repositories str

Comma-separated list of repository names

 required
Example

GET /api/repository/project/json?repositories=tech-radar,another-repo

Expects
  • 200 status code
  • Data for all requested repositories
  • Metadata containing all requested repository names
  • Aggregated statistics across all repositories
Source code in testing/backend/test_main.py 
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
def test_repository_project_json_multiple_repos():
    """Test the repository project JSON endpoint with multiple repositories.

    This test verifies that the endpoint correctly handles requests for
    multiple repositories in a single call. It checks that all requested
    repositories are processed and included in the response.

    Parameters:
        repositories (str): Comma-separated list of repository names

    Example:
        GET /api/repository/project/json?repositories=tech-radar,another-repo

    Expects:
        - 200 status code
        - Data for all requested repositories
        - Metadata containing all requested repository names
        - Aggregated statistics across all repositories
    """
    params = {
        "repositories": "tech-radar,another-repo"
    }
    response = requests.get(
        f"{BASE_URL}/api/repository/project/json", params=params, timeout=10)
    assert response.status_code == 200
    data = response.json()

    # Verify the requested repos are in metadata
    assert len(data["metadata"]["requested_repos"]) == 2
    assert "tech-radar" in data["metadata"]["requested_repos"]
    assert "another-repo" in data["metadata"]["requested_repos"]

Tech Radar Update Tests

These tests are located in test_review.py and verify the review API endpoints.

Missing Entries

Tests handling of missing entries data:

Test the tech radar update endpoint with missing entries.

This test verifies that the endpoint correctly handles requests with missing entries data by returning an appropriate error response.

Endpoint

POST /review/api/tech-radar/update

Expects
  • 400 status code
  • JSON response with error message
  • Error message indicating invalid or missing title
Source code in testing/backend/test_review.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
def test_tech_radar_update_no_entries():
    """Test the tech radar update endpoint with missing entries.

    This test verifies that the endpoint correctly handles requests with
    missing entries data by returning an appropriate error response.

    Endpoint:
        POST /review/api/tech-radar/update

    Expects:
        - 400 status code
        - JSON response with error message
        - Error message indicating invalid or missing title
    """
    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update", json={}, timeout=10)
    assert response.status_code == 400
    data = response.json()
    assert "error" in data
    assert data["error"] == "Invalid or empty entries data"

Partial Updates

Tests processing of partial updates:

Test the tech radar update endpoint with a partial update.

This test verifies that the endpoint correctly processes updates when provided with the complete tech radar structure.

Endpoint

POST /review/api/tech-radar/update

Test Data
  • Complete tech radar structure
  • Valid entries with all required fields
Expects
  • 200 status code
  • Successful update of entries
  • Correct structure in stored data
Source code in testing/backend/test_review.py 
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
def test_tech_radar_update_partial():
    """Test the tech radar update endpoint with a partial update.

    This test verifies that the endpoint correctly processes updates
    when provided with the complete tech radar structure.

    Endpoint:
        POST /review/api/tech-radar/update

    Test Data:
        - Complete tech radar structure
        - Valid entries with all required fields

    Expects:
        - 200 status code
        - Successful update of entries
        - Correct structure in stored data
    """
    random_number = random.randint(100, 1000)
    test_data = {
        "entries": [
            {
                "id": "test-entry-partial-1",
                "title": "Test Entry Partial 1",
                "description": "Languages",
                "key": "test1",
                "url": "#",
                "quadrant": "1",
                "timeline": [
                    {
                        "moved": 0,
                        "ringId": "ignore",
                        "date": "2000-01-01",
                        "description": f"For testing purposes [CASE:{random_number}:1]"
                    }
                ],
                "links": []
            }
        ]
    }

    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json=test_data,
        timeout=10
    )
    assert response.status_code == 200

    # Verify the updates
    get_response = requests.get(f"{BASE_URL}/api/tech-radar/json", timeout=10)
    assert get_response.status_code == 200
    updated_data = get_response.json()

    # Verify our entry exists and is correct
    updated_entries = {entry["id"]: entry for entry in updated_data["entries"]}
    assert "test-entry-partial-1" in updated_entries
    assert updated_entries["test-entry-partial-1"]["timeline"][0]["ringId"] == "ignore"
    assert updated_entries["test-entry-partial-1"]["quadrant"] == "1"

Invalid Entries

Tests validation of invalid entries:

Test the tech radar update endpoint with invalid entries format.

This test verifies that the endpoint correctly handles requests with invalid entries data format by returning an appropriate error response.

Endpoint

POST /review/api/tech-radar/update

Test Data
  • Invalid entries format
  • Missing required fields
  • Malformed data structures
Expects
  • 400 status code
  • Error message for invalid data
  • No changes to existing entries
Source code in testing/backend/test_review.py 
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
def test_tech_radar_update_invalid_entries():
    """Test the tech radar update endpoint with invalid entries format.

    This test verifies that the endpoint correctly handles requests with
    invalid entries data format by returning an appropriate error response.

    Endpoint:
        POST /review/api/tech-radar/update

    Test Data:
        - Invalid entries format
        - Missing required fields
        - Malformed data structures

    Expects:
        - 400 status code
        - Error message for invalid data
        - No changes to existing entries
    """
    # Test with missing title
    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json={"entries": "not_an_array"},
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid or empty entries data"

Valid Structure

Tests updating the Tech Radar with valid data:

Test the tech radar update endpoint with valid complete structure.

This test verifies that the endpoint correctly processes a complete tech radar update with valid structure for all components.

Endpoint

POST /review/api/tech-radar/update

Test Data
  • Valid title
  • Valid quadrants with required fields
  • Valid rings with required fields
  • Valid entries with required fields
Expects
  • 200 status code
  • Successful update confirmation
  • Correct structure in stored data
Source code in testing/backend/test_review.py 
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
def test_tech_radar_update_valid_structure():
    """Test the tech radar update endpoint with valid complete structure.

    This test verifies that the endpoint correctly processes a complete
    tech radar update with valid structure for all components.

    Endpoint:
        POST /review/api/tech-radar/update

    Test Data:
        - Valid title
        - Valid quadrants with required fields
        - Valid rings with required fields
        - Valid entries with required fields

    Expects:
        - 200 status code
        - Successful update confirmation
        - Correct structure in stored data
    """
    random_number = random.randint(100, 1000)
    test_data = {
        "entries": [
            {
                "id": "test-entry-1",
                "title": "Test Entry 1",
                "description": "Languages",
                "key": "test1",
                "url": "#",
                "quadrant": "1",
                "timeline": [
                    {
                        "moved": 0,
                        "ringId": "ignore",
                        "date": "2000-01-01",
                        "description": f"For testing purposes [CASE:{random_number}:2]"
                    }
                ],
                "links": []
            }
        ]
    }

    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json=test_data,
        timeout=10
    )
    assert response.status_code == 200
    assert response.json()["message"] == "Tech radar updated successfully"

    # Verify the update
    get_response = requests.get(f"{BASE_URL}/api/tech-radar/json", timeout=10)
    assert get_response.status_code == 200
    updated_data = get_response.json()

    # Verify entry structure
    entries = updated_data["entries"]
    test_entry = next(
        (entry for entry in entries if entry["id"] == "test-entry-1"), None)
    assert test_entry is not None, "No entry with id 'test-entry-1' found"
    assert str(
        random_number) in test_entry["timeline"][0]["description"], "Entry with id 'test-entry-1' does not have the expected description"

Invalid Structure

Tests the endpoint's handling of invalid data structures:

Test the tech radar update endpoint with invalid structure.

This test verifies that the endpoint correctly validates the complete structure of the tech radar data, including title, quadrants, rings, and entries.

Endpoint

POST /review/api/tech-radar/update

Test Data
  • Missing title
  • Invalid quadrants structure
  • Invalid rings structure
  • Invalid entries structure
Expects
  • 400 status code for each invalid case
  • Appropriate error messages
  • No changes to existing data
Source code in testing/backend/test_review.py 
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
def test_tech_radar_update_invalid_structure():
    """Test the tech radar update endpoint with invalid structure.

    This test verifies that the endpoint correctly validates the complete
    structure of the tech radar data, including title, quadrants, rings,
    and entries.

    Endpoint:
        POST /review/api/tech-radar/update

    Test Data:
        - Missing title
        - Invalid quadrants structure
        - Invalid rings structure
        - Invalid entries structure

    Expects:
        - 400 status code for each invalid case
        - Appropriate error messages
        - No changes to existing data
    """
    # Test missing title
    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json={
            "quadrants": [],
            "rings": [],
            "entries": []
        },
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid or empty entries data"

    # Test invalid quadrants
    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json={
            "title": "Test Radar",
            "quadrants": [{"invalid": "structure"}],
            "rings": [],
            "entries": []
        },
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid or empty entries data"

    # Test invalid rings
    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json={
            "title": "Test Radar",
            "quadrants": [{"id": "1", "name": "Test"}],
            "rings": [{"invalid": "structure"}],
            "entries": []
        },
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid or empty entries data"

Invalid References

Tests validation of references between entries and quadrants/rings:

Test the tech radar update endpoint with invalid references.

This test verifies that the endpoint correctly validates references between entries and their quadrants/rings.

Endpoint

POST /review/api/tech-radar/update

Test Data
  • Entry with invalid quadrant reference
  • Entry with invalid ring reference
  • Entry with missing required fields
Expects
  • 400 status code
  • Appropriate error messages
  • No changes to existing data
Source code in testing/backend/test_review.py 
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
def test_tech_radar_update_invalid_references():
    """Test the tech radar update endpoint with invalid references.

    This test verifies that the endpoint correctly validates references
    between entries and their quadrants/rings.

    Endpoint:
        POST /review/api/tech-radar/update

    Test Data:
        - Entry with invalid quadrant reference
        - Entry with invalid ring reference
        - Entry with missing required fields

    Expects:
        - 400 status code
        - Appropriate error messages
        - No changes to existing data
    """
    test_data = {
        "title": "ONS Tech Radar",
        "quadrants": [
            {"id": "1", "name": "Languages"}
        ],
        "rings": [
            {"id": "adopt", "name": "ADOPT", "color": "#008a00"}
        ],
        "entries": [
            {
                "id": "test-entry",
                "title": "Test Entry",
                "quadrant": "invalid",  # Invalid quadrant reference
                "timeline": [
                    {
                        "moved": 0,
                        "ringId": "invalid",  # Invalid ring reference
                        "date": "2024-03",
                        "description": "Test"
                    }
                ]
            }
        ]
    }

    response = requests.post(
        f"{BASE_URL}/review/api/tech-radar/update",
        json=test_data,
        timeout=10
    )
    assert response.status_code == 400
    assert "Invalid entry structure" in response.json()["error"]

Admin API Tests

These tests are located in test_admin.py and verify the admin API endpoints.

Tests retrieving banner messages:

Test the admin banners endpoint for retrieving banner messages.

This test verifies that the endpoint correctly returns banner messages from the S3 bucket. It checks the structure of the response and ensures the messages array is present.

Endpoint

GET /admin/api/banners

Expects
  • 200 status code
  • JSON response containing a messages array
  • Valid structure that can be parsed by the admin UI
Source code in testing/backend/test_admin.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
def test_admin_banner_get():
    """Test the admin banners endpoint for retrieving banner messages.

    This test verifies that the endpoint correctly returns banner messages
    from the S3 bucket. It checks the structure of the response and ensures
    the messages array is present.

    Endpoint:
        GET /admin/api/banners

    Expects:
        - 200 status code
        - JSON response containing a messages array
        - Valid structure that can be parsed by the admin UI
    """
    response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    assert response.status_code == 200
    data = response.json()

    # Verify the response structure
    assert "messages" in data
    assert isinstance(data["messages"], list)

Tests creating new banner messages:

Test the admin banners update endpoint.

This test verifies that the endpoint correctly processes banner updates with valid data and saves them to the S3 bucket.

Endpoint

POST /admin/api/banners/update

Test Data
  • Valid banner message
  • Array of pages where the banner should appear
  • Optional banner title and type
Expects
  • 200 status code
  • Success message confirming the banner was added
  • Banner should be retrievable in subsequent GET requests
Source code in testing/backend/test_admin.py 
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
def test_admin_banner_update():
    """Test the admin banners update endpoint.

    This test verifies that the endpoint correctly processes banner 
    updates with valid data and saves them to the S3 bucket.

    Endpoint:
        POST /admin/api/banners/update

    Test Data:
        - Valid banner message
        - Array of pages where the banner should appear
        - Optional banner title and type

    Expects:
        - 200 status code
        - Success message confirming the banner was added
        - Banner should be retrievable in subsequent GET requests
    """
    test_data = {
        "banner": {
            "message": "Test Banner Message",
            "title": "Test Banner",
            "type": "info",
            "pages": ["radar", "statistics"],
            "show": True
        }
    }

    response = requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json=test_data,
        timeout=10
    )
    assert response.status_code == 200
    data = response.json()
    assert "message" in data
    assert data["message"] == "Banner added successfully"

    # Verify the banner was added
    get_response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    assert get_response.status_code == 200
    get_data = get_response.json()
    assert "messages" in get_data

    # Find the added banner by message
    added_banner = next((banner for banner in get_data["messages"]
                         if banner["message"] == "Test Banner Message"), None)
    assert added_banner is not None
    assert added_banner["title"] == "Test Banner"
    assert added_banner["type"] == "info"
    assert "radar" in added_banner["pages"]
    assert "statistics" in added_banner["pages"]
    assert added_banner["show"] is True

Tests validation of banner creation requests:

Test the admin banners update endpoint with invalid data.

This test verifies that the endpoint correctly validates banner data and returns appropriate error responses for invalid inputs.

Endpoint

POST /admin/api/banners/update

Test Data
  • Missing message field
  • Empty pages array
  • Malformed banner object
Expects
  • 400 status code
  • Error message indicating invalid banner data
Source code in testing/backend/test_admin.py 
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
def test_admin_banner_update_invalid():
    """Test the admin banners update endpoint with invalid data.

    This test verifies that the endpoint correctly validates banner data
    and returns appropriate error responses for invalid inputs.

    Endpoint:
        POST /admin/api/banners/update

    Test Data:
        - Missing message field
        - Empty pages array
        - Malformed banner object

    Expects:
        - 400 status code
        - Error message indicating invalid banner data
    """
    # Test with missing message
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json={"banner": {"pages": ["radar"]}},
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid banner data"

    # Test with empty pages array
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json={"banner": {"message": "Test", "pages": []}},
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid banner data"

    # Test with malformed request body
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json={"not_banner": {}},
        timeout=10
    )
    assert response.status_code == 400
    assert response.json()["error"] == "Invalid banner data"

Tests toggling banner visibility:

Test the admin banners toggle endpoint.

This test verifies that the endpoint correctly toggles banner visibility by updating the 'show' property in the S3 bucket.

Endpoint

POST /admin/api/banners/toggle

Test Data
  • Valid index of an existing banner
  • New visibility state
Expects
  • 200 status code
  • Success message confirming the visibility update
  • Banner visibility should be updated in subsequent GET requests
Source code in testing/backend/test_admin.py 
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
def test_admin_banner_toggle():
    """Test the admin banners toggle endpoint.

    This test verifies that the endpoint correctly toggles banner visibility
    by updating the 'show' property in the S3 bucket.

    Endpoint:
        POST /admin/api/banners/toggle

    Test Data:
        - Valid index of an existing banner
        - New visibility state

    Expects:
        - 200 status code
        - Success message confirming the visibility update
        - Banner visibility should be updated in subsequent GET requests
    """
    # First, add a test banner
    test_data = {
        "banner": {
            "message": "Toggle Test Banner",
            "pages": ["radar"],
            "show": True
        }
    }

    # Add the banner
    requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json=test_data,
        timeout=10
    )

    # Get all banners
    get_response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    assert get_response.status_code == 200
    banners = get_response.json()["messages"]

    # Find the index of our test banner
    test_banner_index = next((i for i, banner in enumerate(banners)
                             if banner["message"] == "Toggle Test Banner"), None)
    assert test_banner_index is not None

    # Toggle the banner visibility
    toggle_data = {
        "index": test_banner_index,
        "show": False
    }

    response = requests.post(
        f"{BASE_URL}/admin/api/banners/toggle",
        json=toggle_data,
        timeout=10
    )
    assert response.status_code == 200
    data = response.json()
    assert data["message"] == "Banner visibility updated successfully"

    # Verify the banner was toggled
    get_response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    updated_banners = get_response.json()["messages"]
    assert updated_banners[test_banner_index]["show"] is False

Tests validation of banner toggle requests:

Test the admin banners toggle endpoint with invalid data.

This test verifies that the endpoint correctly validates toggle data and returns appropriate error responses for invalid inputs.

Endpoint

POST /admin/api/banners/toggle

Test Data
  • Invalid index (non-numeric)
  • Out of range index
  • Missing index
Expects
  • 400 status code
  • Error message indicating invalid index
Source code in testing/backend/test_admin.py 
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
def test_admin_banner_toggle_invalid():
    """Test the admin banners toggle endpoint with invalid data.

    This test verifies that the endpoint correctly validates toggle data
    and returns appropriate error responses for invalid inputs.

    Endpoint:
        POST /admin/api/banners/toggle

    Test Data:
        - Invalid index (non-numeric)
        - Out of range index
        - Missing index

    Expects:
        - 400 status code
        - Error message indicating invalid index
    """
    # Test with string index
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/toggle",
        json={"index": "not-a-number", "show": True},
        timeout=10
    )
    assert response.status_code == 400
    assert "Invalid banner index" in response.json()["error"]

    # Test with missing index
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/toggle",
        json={"show": True},
        timeout=10
    )
    assert response.status_code == 400
    assert "Invalid banner index" in response.json()["error"]

Tests deleting banner messages:

Test the admin banners delete endpoint.

This test verifies that the endpoint correctly deletes banners from the S3 bucket based on their index.

Endpoint

POST /admin/api/banners/delete

Test Data
  • Valid index of an existing banner
Expects
  • 200 status code
  • Success message confirming deletion
  • Banner should be removed in subsequent GET requests
Source code in testing/backend/test_admin.py 
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
def test_admin_banner_delete():
    """Test the admin banners delete endpoint.

    This test verifies that the endpoint correctly deletes banners
    from the S3 bucket based on their index.

    Endpoint:
        POST /admin/api/banners/delete

    Test Data:
        - Valid index of an existing banner

    Expects:
        - 200 status code
        - Success message confirming deletion
        - Banner should be removed in subsequent GET requests
    """
    # First, add a test banner to delete
    test_data = {
        "banner": {
            "message": "Delete Test Banner",
            "pages": ["radar"]
        }
    }

    # Add the banner
    requests.post(
        f"{BASE_URL}/admin/api/banners/update",
        json=test_data,
        timeout=10
    )

    # Get all banners
    get_response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    assert get_response.status_code == 200
    banners = get_response.json()["messages"]

    # Find the index of our test banner
    test_banner_index = next((i for i, banner in enumerate(banners)
                             if banner["message"] == "Delete Test Banner"), None)
    assert test_banner_index is not None

    # Delete the banner
    delete_data = {
        "index": test_banner_index
    }

    response = requests.post(
        f"{BASE_URL}/admin/api/banners/delete",
        json=delete_data,
        timeout=10
    )
    assert response.status_code == 200
    data = response.json()
    assert data["message"] == "Banner deleted successfully"

    # Verify the banner was deleted
    get_response = requests.get(f"{BASE_URL}/admin/api/banners", timeout=10)
    updated_banners = get_response.json()["messages"]

    # The banner should no longer exist
    deleted_banner = next((banner for banner in updated_banners
                          if banner["message"] == "Delete Test Banner"), None)
    assert deleted_banner is None

Tests validation of banner deletion requests:

Test the admin banners delete endpoint with invalid data.

This test verifies that the endpoint correctly validates delete data and returns appropriate error responses for invalid inputs.

Endpoint

POST /admin/api/banners/delete

Test Data
  • Invalid index (non-numeric)
  • Out of range index
  • Missing index
Expects
  • 400 status code
  • Error message indicating invalid index
Source code in testing/backend/test_admin.py 
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
def test_admin_banner_delete_invalid():
    """Test the admin banners delete endpoint with invalid data.

    This test verifies that the endpoint correctly validates delete data
    and returns appropriate error responses for invalid inputs.

    Endpoint:
        POST /admin/api/banners/delete

    Test Data:
        - Invalid index (non-numeric)
        - Out of range index
        - Missing index

    Expects:
        - 400 status code
        - Error message indicating invalid index
    """
    # Test with string index
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/delete",
        json={"index": "not-a-number"},
        timeout=10
    )
    assert response.status_code == 400
    assert "Invalid banner index" in response.json()["error"]

    # Test with missing index
    response = requests.post(
        f"{BASE_URL}/admin/api/banners/delete",
        json={},
        timeout=10
    )
    assert response.status_code == 400
    assert "Invalid banner index" in response.json()["error"]

Tests the banner message endpoints for retrieving active and all banners:

Test the banner message endpoints.

This test verifies that both banner endpoints (/api/banners and /api/banners/all) return the expected response structure and properly filter active banners.

Tests
  • /api/banners endpoint returns active banners only
  • /api/banners/all endpoint returns all banners
  • Both endpoints handle missing messages.json gracefully
  • Response structure is consistent and valid
Source code in testing/backend/test_main.py 
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
def test_banner_endpoints():
    """Test the banner message endpoints.

    This test verifies that both banner endpoints (/api/banners and /api/banners/all)
    return the expected response structure and properly filter active banners.

    Tests:
        - /api/banners endpoint returns active banners only
        - /api/banners/all endpoint returns all banners
        - Both endpoints handle missing messages.json gracefully
        - Response structure is consistent and valid
    """
    # Test /api/banners endpoint
    response = requests.get(f"{BASE_URL}/api/banners", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert "messages" in data
    assert isinstance(data["messages"], list)

    # Verify active banners only
    for banner in data["messages"]:
        assert banner.get("show", True) is True

    # Test /api/banners/all endpoint
    response = requests.get(f"{BASE_URL}/api/banners/all", timeout=10)
    assert response.status_code == 200
    data = response.json()
    assert "messages" in data
    assert isinstance(data["messages"], list)

    # Test error case with non-existent endpoint
    response = requests.get(f"{BASE_URL}/api/banners/nonexistent", timeout=10)
    assert response.status_code in [404, 500]

The banner endpoint tests verify: - Active banners are correctly filtered in the /api/banners endpoint - All banners (active and inactive) are returned by /api/banners/all - Missing messages.json is handled gracefully - Response structure is consistent and valid - Error cases are properly handled with appropriate status codes

Error Handling Tests

Invalid Endpoints

Tests the server's response to non-existent endpoints:

Test error handling for invalid endpoints.

This test verifies that the server properly handles requests to non-existent endpoints by returning appropriate error status codes.

Example

GET /api/nonexistent

Expects
  • Either 404 (Not Found) or 500 (Internal Server Error) status code
  • Proper error handling for invalid routes
Source code in testing/backend/test_main.py 
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
def test_invalid_endpoint():
    """Test error handling for invalid endpoints.

    This test verifies that the server properly handles requests to
    non-existent endpoints by returning appropriate error status codes.

    Example:
        GET /api/nonexistent

    Expects:
        - Either 404 (Not Found) or 500 (Internal Server Error) status code
        - Proper error handling for invalid routes
    """
    response = requests.get(f"{BASE_URL}/api/nonexistent", timeout=10)
    assert response.status_code in [404, 500]  # Either is acceptable

Invalid Parameters

Tests the server's handling of invalid parameter values:

Test the JSON endpoint's handling of invalid date parameters.

This test verifies that the endpoint gracefully handles invalid datetime parameters without failing. It should ignore the invalid date and return unfiltered results.

Parameters:

Name Type Description Default
datetime str

An invalid datetime string

 required
Example

GET /api/json?datetime=invalid-date

Expects
  • 200 status code (graceful handling)
  • Null filter_date in metadata
  • Valid response with unfiltered stats
  • Complete language statistics
Source code in testing/backend/test_main.py 
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
def test_json_endpoint_invalid_date():
    """Test the JSON endpoint's handling of invalid date parameters.

    This test verifies that the endpoint gracefully handles invalid datetime
    parameters without failing. It should ignore the invalid date and return
    unfiltered results.

    Parameters:
        datetime (str): An invalid datetime string

    Example:
        GET /api/json?datetime=invalid-date

    Expects:
        - 200 status code (graceful handling)
        - Null filter_date in metadata
        - Valid response with unfiltered stats
        - Complete language statistics
    """
    response = requests.get(f"{BASE_URL}/api/json",
                            params={"datetime": "invalid-date"}, timeout=10)
    assert response.status_code == 200  # Backend handles invalid dates gracefully
    data = response.json()
    assert data["metadata"]["filter_date"] is None
    assert "stats" in data
    assert "language_statistics" in data

Test Execution Flow

The backend tests follow this general execution flow:

  1. Setup: Configure the test environment and parameters
  2. Request: Make an HTTP request to the target endpoint
  3. Validation: Assert that the response status code is as expected
  4. Data Verification: Assert that the response data structure is correct
  5. Content Verification: Assert that the response data contains the expected values

Integration with Frontend Utilities

These backend tests validate the same endpoints that are used by the frontend utilities:

  1. Project Data Utility: The test_csv_endpoint() test validates the endpoint used by fetchCSVFromS3()
  2. Repository Data Utility: The repository project tests validate the endpoint used by fetchRepositoryData()
  3. Tech Radar Data Utility: The test_tech_radar_json_endpoint() test validates the endpoint used by fetchTechRadarJSONFromS3()
  4. Admin Utilities: The admin API tests validate the endpoints used by the admin interface for banner management