feat: detect incorrect integration test functions (#8352)
Some checks are pending
/ release (push) Waiting to run
testing-integration / test-unit (push) Waiting to run
testing-integration / test-sqlite (push) Waiting to run
testing / backend-checks (push) Waiting to run
testing / frontend-checks (push) Waiting to run
testing / test-unit (push) Blocked by required conditions
testing / test-e2e (push) Blocked by required conditions
testing / test-remote-cacher (redis) (push) Blocked by required conditions
testing / test-remote-cacher (valkey) (push) Blocked by required conditions
testing / test-remote-cacher (garnet) (push) Blocked by required conditions
testing / test-remote-cacher (redict) (push) Blocked by required conditions
testing / test-mysql (push) Blocked by required conditions
testing / test-pgsql (push) Blocked by required conditions
testing / test-sqlite (push) Blocked by required conditions
testing / security-check (push) Blocked by required conditions

- I have seen multiple times where a test function tries to prepare the
testing environment more than once, this can lead to bugs and false
positives of testing code. I would attribute this to lack of
documentation on how to write integration tests.
- To detect such cases, keep track when we are in a prepared test
environment and fail when some testing code is tries to once again
prepare the test environment.
- The message is logged to the function call that is requesting to
prepare the test environment, for example: `change_default_branch_test.go:19: Cannot prepare a test environment if you are already in a test environment. This is a bug in your testing code.`

A example of what this will be able to catch, 6226f464ce:

```go
func TestFoo(t *testing.T) {
	defer PrepareTestEnv(t)()

	t.Run("Bar", func(t *testing.T) {
		defer PrepareTestEnv(t)() // Should very likely be PrintCurrentTest
	})
}
```

```go
func TestBar(t *testing.T) {
	onGiteaRun(t, func(t *testing.T, _ *url.URL) {
		defer PrepareTestEnv(t)() // Already called by onGiteaRun.
	})
}
```

```go
func TestFooBar(t *testing.T) {
	defer PrepareTestEnv(t)() // This will be called by onGiteaRun later on and very unlikely to do this before the call to onGiteaRun.
	onGiteaRun(t, func(t *testing.T, _ *url.URL) {
		// [...]
	})
}
```

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8352
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
This commit is contained in:
Gusted 2025-06-29 23:09:29 +02:00 committed by Earl Warren
parent c57dea336c
commit 4927d4ee3d
7 changed files with 47 additions and 32 deletions

View file

@ -89,7 +89,6 @@ func TestAPIPullUpdateByRebase(t *testing.T) {
func TestAPIViewUpdateSettings(t *testing.T) {
onGiteaRun(t, func(t *testing.T, giteaURL *url.URL) {
defer tests.PrepareTestEnv(t)()
// Create PR to test
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
org26 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 26})
@ -136,7 +135,6 @@ func TestViewPullUpdateByRebase(t *testing.T) {
func testViewPullUpdate(t *testing.T, updateStyle string) {
defer test.MockVariableValue(&setting.Repository.PullRequest.DefaultUpdateStyle, updateStyle)()
defer tests.PrepareTestEnv(t)()
// Create PR to test
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2})
org26 := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 26})