It's unsafe with many makefiles that use .d files.
However, in case of FreeBSD ports, it should be safe, as .d files are not needed there. They're only for compiling after updating a source tree with still lying around .o files.
If you always do make clean before make, -j is also always safe.
Basically, if you're using -j, you may get weird errors when working in a non-clean source tree. make clean before recompiling again, to see if that makes the problem go away then.