(Edit: I mixed up OpenBSD and NetBSD while writing this issue. This issue applies only to NetBSD. I've edited the issue accordingly.)
Summary
If I mock a function in NetBSD, then my mock code runs instead of the real code, but my custom exit code is ignored.
Examples
Basic example
The following test fails on NetBSD. On my own Debian machine, this test passes.
#!/bin/sh
Describe "my tests"
It "mocks functions"
Mock mkdir
return 1
End
When run mkdir -p foo
The status should be failure
End
End
Error message (click to reveal)
Running: /bin/sh [sh 20220122 BUILD:20241216130811Z]
F
Examples:
1) my tests mocks functions
When run mkdir -p foo
1.1) The status should be failure
expected: failure (non-zero)
got: success (zero) [status: 0]
# example_spec.sh:9
Finished in 0.05 seconds (user: 0.03s, sys: 0.03s) [time: external-time]
1 example, 1 failure
Failure examples / Errors: (Listed here affect your suite's status)
shellspec example_spec.sh:3 # 1) my tests mocks functions FAILED
Mock code runs, but exit code is ignored
If I add an echo to the mock's code, then the echo is reported, showing that the mock does indeed run. Moreover, the directory foo isn't actually created, showing that the real mkdir isn't being invoked either.
The test still fails, though. On Debian, this test succeeds.
#!/bin/sh
Describe "my tests"
It "mocks functions"
Mock mkdir
echo foobar
return 1
End
When run mkdir -p foo
The status should be failure
End
End
Error message (click to reveal)
Running: /bin/sh [sh 20220122 BUILD:20241216130811Z]
F
Examples:
1) my tests mocks functions
When run mkdir -p foo
1.1) The status should be failure
expected: failure (non-zero)
got: success (zero) [status: 0]
# example_spec.sh:10
1.2) WARNING: There was output to stdout but not found expectation
stdout:foobar
# example_spec.sh:3-11
Finished in 0.05 seconds (user: 0.03s, sys: 0.02s) [time: external-time]
1 example, 1 failure
Failure examples / Errors: (Listed here affect your suite's status)
shellspec example_spec.sh:3 # 1) my tests mocks functions FAILED
Custom functions are similarly affected
It's not just existing programs for which mocks fail. When mocking a custom function, the body is executed, but the exit code is ignored.
#!/bin/sh
Describe "my tests"
It "mocks functions"
Mock foobar
echo foobar
return 1
End
When run foobar
The status should be failure
End
End
Error message (click to reveal)
Running: /bin/sh [sh 20220122 BUILD:20241216130811Z]
F
Examples:
1) my tests mocks functions
When run foobar
1.1) The status should be failure
expected: failure (non-zero)
got: success (zero) [status: 0]
# example_spec.sh:10
1.2) WARNING: There was output to stdout but not found expectation
stdout:foobar
# example_spec.sh:3-11
Finished in 0.05 seconds (user: 0.03s, sys: 0.02s) [time: external-time]
1 example, 1 failure
Failure examples / Errors: (Listed here affect your suite's status)
shellspec example_spec.sh:3 # 1) my tests mocks functions FAILED
Potential workaround
Replacing return 1 with exit 1 in the mock definition resolves the issue on NetBSD. However, I'm unsure of whether this has any negative side effects, such as inadvertently aborting the mock's process and preventing cleanup, or something like that.
Version information
(Edit: I mixed up OpenBSD and NetBSD while writing this issue. This issue applies only to NetBSD. I've edited the issue accordingly.)
Summary
If I mock a function in NetBSD, then my mock code runs instead of the real code, but my custom exit code is ignored.
Examples
Basic example
The following test fails on NetBSD. On my own Debian machine, this test passes.
Error message (click to reveal)
Mock code runs, but exit code is ignored
If I add an
echoto the mock's code, then the echo is reported, showing that the mock does indeed run. Moreover, the directoryfooisn't actually created, showing that the realmkdirisn't being invoked either.The test still fails, though. On Debian, this test succeeds.
Error message (click to reveal)
Custom functions are similarly affected
It's not just existing programs for which mocks fail. When mocking a custom function, the body is executed, but the exit code is ignored.
Error message (click to reveal)
Potential workaround
Replacing
return 1withexit 1in the mock definition resolves the issue on NetBSD. However, I'm unsure of whether this has any negative side effects, such as inadvertently aborting the mock's process and preventing cleanup, or something like that.Version information
.shellspecfile.--shell sh, as suggested in "Not found specified shell" in OpenBSD #291 for a different issue that was on OpenBSD, does not affect this issue in any way.