You have to make it go somehow. I don’t think there are any serious alternatives to a mouse that uses batteries for stored energy and some kind of electric motor for the driving force.
There are some basic (ie simple enough for me) physics to look at and then the choice between two common types of electric motor. I have only bothered with stuff on steppers and DC motors as these will do the job and present enough complexity to keep anyone happy. If you want to try synchronous, AC or brushless motors then you carry on. If you want steam or clockwork then you are already in trouble. Remember that there is a rule about non-polluting drive systems.
Essentially, you want a driving unit which you can easily and accurately control in terms of acceleration, speed and position. It must be light, small and use as little energy as possible. A minimum of ancillary electronics would be good and it should preferably be easy to mount and cheap.
If you find a motor that genuinely meets these criteria, tell lots of people, they all want to know. In the real world, which of these are you going to forfeit? If you have enough money then that need be all you lose.
For convenience, you might want to choose stepper motors. They are heavy and power hungry, not always good for speed and will need relatively heavy duty driving electronics. However, control is a doddle and mounting could hardly be easier. The first mouse ever to succeed used steppers as do a number of very competitive mice from Korea, Japan and Singapore. Usable steppers can be recovered from old disk drives althought these are not likely to be competitive in modern competition use.
For performance, DC motors are the way to go. They can be small and light, the electronics are fairly simple and the power requirements quite modest. If you want to solve the motion control problem properly, you will need more clever software and a processor that can do maths (fast multiplies in particular). Mounting and finding suitable gearboxes can be a pain. High quality, small DC motors are not cheap if purchased new but are available on the surplus market. Multi-wheel mice may need six or eight DC motors.
Before it occurs to you, converted radio control servos are not necessarily up to the job if you simply disconnect the output pot for continuous rotation. They don’t go fast enough and the motors in them are not designed for a long life of continuous running. Users report motors can burn out after a few tens of hours. If you want to mess with the gearing, however, and are prepared to put up with a potentially reduced service life, they are very convenient. There are a number of competitive mice that use this drive system.
The new style Lego motor looks promising if you can only fit suitable sensors for speed and position. It can do about 480rpm under no load with a 12 volt drive. Not really competition stuff but certainly convenient. If you want to risk it, you can take them apart and fit a reflective sensor to the rotor for speed control. In the picture below, you will see a hole drilled in the side of the case. behind that is the motor’s rotor. Careful dissassemby let me paint a number of white spots on the rotor. A reflective sensor looks in through the hole to give speed control.
In the end, you will have to pick. Base your decision on what you can get and what you can control. For a first mouse, be happy to get it to work at all. Trust me, you will be. I have chosen steppers because they are easy, adequate and available. I already have plans for a slick DC motor driven beast.