TL;DR the problem was PHP's default memory limit.
We've had a mysterious case of PHPUnit tests failing on Github Actions. How could tests be passing on our local machine, but failing on Github?
To make matters worse, the error message didn't provide any hints. There weren't any errors in the logs, or any indication of what exactly was failing.
Basic fact checks
When the tests pass on one environment but fail on the next, the obvious place to look for is the parity between these environments.
- Are PHP versions the same?
- If we're hitting any resources like DB, cache, etc; are we using exactly the same versions?
In our case, both platforms ran PHP 7.4, and used the same version of Postgres.
Gathering clues
There's a debug mode in PHPUnit. This mode is really helpful as it gives the most insight into what's happening, step by step.
./vendor/bin/phpunit --debug
The output was this:
Seeing that admin_can_update_customer started but didn't end, we naturally assumed we've found the culprit.
Isolate the suspect
As a rule of thumb, your tests should run in isolation and they should never affect other tests if you write them properly. If you make your tests depend on a certain order of execution, they break when you run them one by one (or vice versa). So we thought maybe that's the case here.
./vendor/bin/phpunit --filter=UsersTest
To test this, we asked PHPUnit to run only UsersTest. Nope... All is green ✅
Back to the beginning
After exhausting all the obvious steps, we had to get creative. Why would a process suddenly stop, fail, and print no errors whatsoever? Why would the same process work fine on the other machine?
It dawned on me to check the memory consumption after running the tests locally. It was 131 MB. Bingo! 🎉
The default PHP memory limit on the docker image we used was 128 MB.
RUN echo 'memory_limit = 512M' >> /usr/local/etc/php/conf.d/docker-php-memlimit.ini;
I've added this to our dockerfile, rebuilt the image and pushed it to Docker Hub. Sure enough, our problem disappeared. Day saved!
P.S. If you're looking to test your Laravel app on Github Actions, check out our Docker image that comes bundled with all the extensions + chromium (to run Dusk).
