The holding device is even more important than the indicator itself. You can purchase magnetic base holders for under $20, but may be you have found these to be awkward to adjust. Here's a typical low-cost holder:
A lever which has the effect of turning the magnetic field on and off. Since the magnetic force is quite powerful, on the order of 50 lbs., turning the field off makes it much easier to remove and adjust the base. Naturally, the base will only clamp onto steel, cast iron or other ferrous metals. Usually we clamp it to the headstock, the ways or the top of the compound.On the back side of the DI is a mounting lug through which a locking bolt is passed to attach the DI to the holder.
The problem with the cheap holders is that it that you must independently adjust the two axes to position the indicator. Especially within the tight confines of the mini-lathe and mill, you will find that getting the indicator in the exact position you want it is often difficult. Maximum flexibility of the holder is thus a great time saver.
Fortunately, holders are available which can be locked in place using a single knob after the indicator is in the desired position.
Even though it lacks the ability to slide the DI along the arm, I have found this holder to be very versatile and much quicker to set up than the cheaper one.