Not Normal, Safe
DEAN: So what are you gonna do? You’re just gonna live some normal, apple pie life? Is that it?
SAM: No. Not normal. Safe.
DEAN: And that’s why you ran away.
SAM: I was just going to college.
Okay, I’m pulling this bit out because there are two, let’s call them, myths in the fandom. 1. that Sam has always wanted a “normal, apple pie” life, and 2. that he runs away from problems. This bit of conversation refutes both of those things. Sam’s desire to get away from hunting isn’t about wanting normal; it’s about wanting to be safe, which he has never felt. Remember that he never had even the four years of a safe, stable home life that Dean did. Also, Sam’s statement that he was “just going to college” indicates that until John said to never come back, he hadn’t intended to stay gone.
From Phantom Traveler:
DEAN: Seriously, are you still having nightmares about Jess?
SAM: Yeah. But it’s not just her. It’s everything. I just forgot, you know? This job. Man, it gets to you.
Dean denies that he the constant fear bothers him as well despite the fact that he sleeps with a hunting knife under his pillow. We know however that Dean has a defense mechanism — he pushes his feeling down. He, as he does in this scene, denies his feelings. Sam doesn’t have those kind of defenses.
What going to Stanford and living with Jess did was make Sam think that he could get away from hunting and the fear that went with that life. Again, from the pilot:
DEAN: Monday. Right. The interview.
DEAN: Yeah, I forgot. You’re really serious about this, aren’t you? You think you’re just going to become some lawyer? Marry your girl?
SAM: Maybe. Why not?
DEAN: Does Jessica know the truth about you? I mean, does she know about the things you’ve done?
SAM: No, and she’s not ever going to know.
DEAN: Well, that’s healthy. You can pretend all you want, Sammy. But sooner or later you’re going to have to face up to who you really are.
SAM: And who’s that?
DEAN: You’re one of us.
SAM: No. I’m not like you. This is not going to be my life.
Of course, we all know now that as fate would have it, Sam was destined to not have a safe life, and Dean was right, Sam is one of them. He is a hunter. Dean’s question here about whether Jess knew about the things that Sam had done is rather chilling. Dean knows his brother. He knows what Sam is capable of even at that young age, but Sam was frightened and idealistic and tragically naïve. He had no idea what supernatural forces far greater than any they had encountered had in store for him.
In season 6, Sam told Samuel Colt that there was no getting out of the life. It may sound sadly cynical, but of course, he’s right. He’d realized by then that it wasn’t Dean that would drag him back into the life; it was the life. He’d seen how it dragged Dean back in when the djinn came for him at Lisa’s house. That’s why Sam’s life with Amelia was just a dream. Even if Don hadn’t returned from the dead so to speak, Dean would have or monsters would have come for Sam. That’s what Sam’s speech to Fred was about when he said that it’s nice to live in a dream world, but reality will always come back and destroy everything.
Sam was still clinging to that dream when he told Dean that after they found the tablets he would stop hunting. Dean was back and for that he was glad, but with Dean the fear returned – not just fear for himself, but fear of losing Dean again. He’d barely survived losing him the last time, and he didn’t think he could again.