Installed the new filter. When I started it up to check for leaks, it was gushing oil. Yikes. I got a lovely 4 x 4 ft puddle of oil on the garage floor out of it though.
Turns out I installed it on top of the old oil filter gasket which had stuck to the block. Caveman says it's not a filter quality issue, so I'll chalk it up to a novice mistake this time.