I had one of those small Timex Sinclaire computers (I can't remember which one). I don't think it used CMOS parts, so the power consumption would have been too high for economical 9V battery use. Do you know how much current it takes? If it's 250mA for example, you might get as much as two hours of use from a 9V alkaline battery, and a lot less per charge from a rechargeable one.
I don't know what kind of TV you could get today that would be tiny. There were pocket TVs (I remember Radio Shack selling them) before smartphones and digital TV, but there probably weren't enough sold and they probably weren't taken care of well enough to still be operational so you could find one on eBay.
What might be a good goal would be to see what you can learn from the packaging of computers like the ZX-81 and apply it to a computer of your own build. You'll want to use CMOS parts of course, not NMOS, and not 74LS. There are lots of shapes and sizes and colors of ABS plastic project boxes available from gobs of manufacturers (just take a look in the
Mouser catalog, or even
Jameco which is much smaller than Mouser). ABS is marvelous material for milling with a Dremell tool and filing and drilling and hot-glueing. Some come with battery compartments for the 9V or AA batteries. I'm sure you won't be spending the tool-up costs for a custom membrane keyboard like the Timexes had, but there are some smaller off-the-shelf ones, and you can use something like the Grayhill series-87 keyswitches which fit neatly into standard prototyping board and their transparent keycaps can be removed and put back to put your own custom-printed labels under them. LCDs come in a gazillion sizes and types, and there are some tiny VGA adapters like I show on the
displays page of my
6502 primer.