There are a number of issues around measuring high voltages that don't exist inside the normal voltmeter, so we need a place to discuss those.
One is that you don't want your measuring device (usually) to draw much power, and as volts go up, for the same current draw from the measured source, power goes up. At some point, this may either strain the source, or your dividing chain's ability to dissipate power, so you are walking a line with this. As you try to go ever farther down in current because of this, the current can become so low at the output of the divider that noise issues crop up there, and you might need a very low current low voltage measuring device to read the result accurately.
Unlike a lot of other situations, at high voltages we have to begin to seriously consider various losses in insulators, which can either raise or lower the result you measure. For example, if there is something covering your series resistor (like fingerprints) the value of some of the large resistors can be reduced significantly, making your result read too high. On the other hand, if there's corona current being drawn from someplace along the series resistor, you reading will be lower than true. And it doesn't take an amount of either you'd think would be a serious issue. Even the resistance of pyrex glass or most any surface in high humidity can significantly affect your reading.
For voltages up to about 40-50kv, a TV service HV probe is mostly likely the way to go. These (B&K) and variations made by Fluke, Hewlett Packard and so on can often be had at hamfests, and they are for the most part, quite well made. They've solved the above problems pretty well up to these voltages, and usually will drive a 10 meg input DVM with minimal error (but beware some Harbor Freight meters that don't really have 10 meg inputs....despite the specs). Most of these are 1000:1, so give 1v per kilovolt, fairly handy with standard meters. Most also have reasonable frequency response at least up to the 10's of khz, which isn't fantastic, but isn't too bad unless you're looking for RF, and the word here is -- test it with a scope and known waveform from a frequency generator.
When you get to really high voltages, over 50k or so, the issues mentioned above become quite serious and it becomes hard to make up a divider chain that will reliably tell the truth to better than 10% (and fairly hard to get to 10%). At this point, all the issues above really start to kick in, and precision high value resistors aren't cheap at all, if you can even find them. Vishay and a few others make them, but most retailers don't carry them, and they're not cheap at all in any event. Most outfits that specialize in high voltage things like power supplies will sell you a divider that's guaranteed accurate, but the price may be a major shock. It's not that they are ripping you off, it's a genuinely hard problem to solve, especially when you have to expect the user isn't an expert (or they'd make their own, after all).
We have made a few successful probes here. We use a whole bunch of 10 meg resistors, usually 1/2 watt precision ones (not real cheap) that will stand off on the order of 1kv each according to ratings, for a max current of 100 ua. That's too much, of course, for most situations, so we don't use them at 1kv each -- we tend towards using more of them to get the current to half that or less, because that's too many watts for a light load on the measured source. For example, for a 100kv "probe" at 100 uA you are talking 10 watts load, which is enough to warm the resistors a good bit, which eventually ruins their not-very-good accuracy. To help with this, and with corona issues, we put them end to end in long quartz or soda-lime glass tubing, usually 3/8" or better, 1/2" stuff, and fill that with mineral oil. you make this chain of resistors, just solder them end to end, clean off the flux(!) and solder them to the center conductor of some common coax.
This is then glued into one end of the tube with silicone RTV (or something better if you have it, but beware stiff/brittle glues that will break easily later on). Once this has dried fully, we then stand the tube on end with the lead (use stranded wire) coming out the end, and fill it most of the way up with very pure mineral oil (we get it at McMaster Carr, it's better than the stuff they call baby oil). This has to be done carefully so as not to get any oil on the glass near the top, so you can seal that end too. We use a little homemade funell that terminates in some 1/8" OD copper tubing that you can stick down in there, and are careful extracting it when done so as not to wipe oil onto the inside of the tubing -- difficult, but crucial, and being very careful means not having to start all over with a new piece of coax (which you know you got oil on) and so on. We leave a bit of air space in the top, so when we seal it with RTV or whatever, there's some room for expansion to take place when the thing warms up. You can then affix some sort of corona ball to the "hot" end if you like or have the thing terminate inside the one on the device (which is going to have one if you're doing 100kv or more), to avoid streamer arcs into the air off sharp points. In some cases, your power supply might be all in oil anyway, which means you can put this whole thing in there with it, and not need the glass at all, which is a lot simpler.
Now we come to frequency accuracy. The way this is done in say, a scope probe (which works here too) is to put X capacity across the 9 meg resistor, and ten times as much across the 1 meg load resistor. As often as not, this doesn't actually take any physical capacitors other than those that exist parasitically anyway. Here, we have a tougher problem. You could either put tiny caps across each of the 10 meg series resistors...not cheap and not easy, with regular 1kv or so caps (if you can find them still) if that part of the chain has less than the required capacity, or try some other way. Here we have put a sleeve over some of the hot end of the probe, connected to the HV end, so as to get some through the glass, oil, etc to the chain.
Of course, you could try to reduce the capacity across the low end resistor, but there's going to be a limit, and any wire is going to add to that, or even the package capacity of an opamp you drive with that end. Depending on the details of your design and how it's realized in practice, you could have to add capacity to either side. We find that with the oil, which has a high dielectric constant, it comes out not too far off with nothing -- good deal if you can get it. Driving into a virtual ground at the bottom might help in some cases -- the inverting input of an opamp that has a feedback resistor, can help a lot. But at 50uA or so full scale currents, it has to be a really good opamp, with very low input currents, or that and drifts are going to drive you crazy. So choose wisely, there are a few I might mention in a later post if asked that seem to work well here. We've not yet tried the techniques Charles W is using on his ion chambers, or those fancy opamps with a built in chopper for stability, but it's quite possible those would do the best. I've been using things like LMC 660's with reasonable results, and live with a little current drift and the lack of HF response, but like always, Your Mileage May Vary. For now, when I'm looking for RF on top of a big DC source, I just place a scope probe a safe distance away from it (all inside a big grounded shield screen) and let the natural capacitive dividier do the job. This isn't quantitative by any means, but more often than not, good enough.
Some people find that for low accuracy situations, they can simply measure the voltage somewhere along the power supply's voltage multiplier. This will read high under load, due to increased drops in the multiplier stack, as well as those due to any ballast resistor (which you most probably need for the application anyway). But maybe you don't need extreme accuracy, and can back-calculate those losses anyway, from measuring current as well -- that's one of those "if it works, why not?" sorts of things.