Lots of really well thought-out replies in this thread, I think pretty much everything's been covered from the basics to specifics to all the stuff that's in between.

I only have an example to throw in from my own gaming experience (because most so far have dealt with either FEAR or HL2 or Silent Hill or otherwise, for good reason) - the first mission as a Marine in AVP2. It might sound silly to some considering the game is mostly action-action, but the first mission was absolutely terrifying.

To start, they covered all the basics. Creepy sound effects, terrifying setting (stuck alone with aliens - and as you later find out a Predator to boot), electric wires popping out of the ceiling at all the wrong moments. But here's the kicker - at least for me - the scariest parts were, as most have described, the ones where the build-up was psychological rather than obvious, in your face. See in the mission, after a cave in, you're stranded from the rest of your team and forced to navigate through an empty giant structure (or your portion of it) all by your lonesome. One of the gadgets you're dispatched for this job is the radar, as seen in Aliens, which goes off if anything starts moving in the nearby vicinity.

Now as this is a full-fledged complex, there's doors and elevators to traverse up the wazoo, and any time you open one, the radar goes off. Call it what you will, but the first time through I didn't pick up on that (mainly because there's simply so many reasons why a radar could potentially go off, such as even a rat or some bug), so every time I opened the elevator and it was moving I'd see the radar go off and I thought that some aliens were just in the room I was about to come into running around, getting into position, and let me tell you I was freaked out. The doors opened and I'd be on full alert, scouring every corner. I guess this is another example of something not behaving as it should, in this case your radar not giving you full, detailed information and making you twitch unnecessarily.

Otherwise, near the end of the mission you're supposed to move some stuff around for a dropship to land on your side to pick you up, and that's no less scary. As I found out first hand, it can be incredibly terrifying to have to do something while you're expecting enemies to pop up and eat your face off. Just the prospect on turning your back to the darkness around you because you HAVE TO can be freaky, and you'll be listening extra hard to spot anything before it comes.

It's been said before in this thread, and it's been summarized much better than anything I could attempt, but this is just another first-hand sort of perspective on some stuff that could make shivers run down my (or your) spine. :)