Knowing What to Change
This is probably the hardest part of overclocking. After you increase your speed and run a test, where do you make changes if it failed?
The first rule is to make sure you're changing as few variables at a time as possible. By this, as I've mentioned before, I mean overclock your CPU independently from your RAM or anything else. Focus on the CPU and then you'll know what to change. The first course of action is to increase the Vcore. I like to use the +/- keys for this, and generally give it "two increases". After that test again. If it fails, you can either do Vcore again or increase the VTT by a similar amount. If you are using an unlocked CPU, you mostly only need to focus on Vcore.
In the BIOS, the Vcore is usually set to Auto by default. To a certain extent, your motherboard and CPU will be able to automatically increase the voltage to your CPU as needed, however, I discourage this practice. As far as I know, it essentially works by reading values off a table and picking what seems to be the right one - in no way does it adjust "on the fly" as required. It's possible that some top end motherboards can do so, but either way you will achieve greater stability by manually adjusting the voltage as required. You can usually set the voltage to "offset" or "manual", where manual shows you the full voltage reading and offset merely shows the amount above or below the stock voltage you are at. Manual is my preference.
If you are at a fairly reasonably high overclock, like 4+ghz or 4.5ghz on a Sandy Bridge, you're probably going to hit a wall. What this means is that as you increase the CPU clock, it will lose efficiency. Eventually a small increase in speed will require a large increase in voltage. If you notice as you go that you start requiring more and more voltage increases to get stable, you are definitely nearing the wall and should consider whether you are happy with your current overclock. I've mentioned before that CPUs do have a voltage limit to them, particularly if you want it to last a while.
You also need to watch your temperature. Higher voltages will produce higher temperatures, and high temps are a detriment to CPU stability and longevity. If you are heating up at an abnormal rate, you might need to check that your CPU cooler is attached properly. If it is, and it's clean, and you have good case airflow, then you reached the limit of that cooler and will either need to just live with the clock you've reached, or buy a better one.
If you are pushing a very high overclock - perhaps for benchmarking - and are not terribly concerned by the voltage and/or temps, then you might find other factors than voltage are hurting your stability. You may need to disable EIST/Speed Step, C States, Cool N Quite (AMD), and other things. If there is an option you are unfamiliar with, it is a good idea to check your motherboard manual and to google for information on it.
Info:
Example of Intel's Tech Specs (i5 750) - View section 7.4 for the VID table, or 7.8 for the absolute max voltage the CPU can handle.
Intel Tech Support - Search your CPU here, and select "Documentation and Guides". Then go to "General Product Information" and select "Technical Documentation".
AMD Tech Support - Search your CPU here, and select "Technical Documents". The select the "Desktop Processor Power and Thermal Data Sheet" - note: this doesn't appear for the FX series at this time.
Setting the RAM
Finally, I hope, you'll have reached a stable CPU overclock. I also hope that you are using a BCLK that will allow you to set your RAM to it's rated speeds. Ideally, you will just select the right multiplier to get the proper speed out of it. You also need to set the voltage to the appropriate amount - many RAM kits are 1.5V but for example I use RAM rated for 1.65V. Just verify your DRAM voltage is right.
If you have the speed and voltage right, there is still one more thing to adjust - the timings. If you see a RAM rating, it is usually something like "1600mhz 1.5V CL9". The CL9 means the CAS Latency = 9. The 9 refers to how many clock cycles it can take. There are, however, many more timings than just CL9. Usually, CL9 actually means 9-9-9-27 which are the 4 most important timings. The first 3 timings are almost always identical, like the former example. They could also be 8-8-8-24, for example. The 4th value is usually just the first 3 added together.
For a lot more detail on that, this is a pretty good article:
Understanding RAM Timings
Suffice it to say that as you SPEED goes up, your TIMINGS do as well. Essentially, any given RAM will run at a particular speed for a particular timing. So if you have 1400mhz RAM at CL7, you might be able to run it at 1600mhz CL8 or 1800mhz CL9. This is not a guarantee, but it's very possible if it is good RAM. Also increasing the voltage can help the RAM run at higher speeds.
Ideally, you will not have to make the RAM run at too high of a speed, but if your CPU overclock gives a solid 1600mhz possibility for the RAM, you might wish to try it by either increasing or decreasing the timings based on what they are rated for at stock speeds. If you can't find your stock settings, check CPU-Z under the Memory tab. It will list JEDEC specs, which are universal, but if it has XMP specs then those are specific to your RAM.
XMP means "Extreme Memory Profile". Technically, anything over 1333mhz is "overclocked". XMP is fine in that it does set your RAM to it's rated speed, but the problem is that sometimes it will mess with your BCLK/FSB. You may want to use it to view the recommended settings, but once you found your CPU's OC settings, do not accidentally undo it all by setting the XMP (of course, you can choose to not save the changes if you want).
After any changes, again, make sure to test. You'll want to use a maximum memory usage test in Intel Burn Test, or even run Memtest86+ or a long Blend test in Prime95.
