Cancer evolution cannot be observed directly in patients, and new methodologies are needed for obtaining a quantitative understanding of this obscure process. We developed and analyzed a stochastic model of malignant transformation in the colon that precisely quantifies the process of colorectal carcinogenesis in patients through loss of tumor suppressors APC and TP53 and gain of the KRAS oncogene. Our study employs experimentally measured mutation rates in the colon and growth advantages provided by driver mutations. We calculate the probability of a colorectal malignancy, the sizes of premalignant lesions, and the order of acquisition of driver mutations during colorectal tumor evolution. We demonstrate that the order of driver events in colorectal cancer is determined primarily by the fitness effects that they provide, rather than their mutation rates.