SD card would certainly be my first thought: you don't necessarily need any shift registers, because you're providing both clock and data, so you can proceed as slowly and irregularly as you like.
There is then the question of what you do with the blocks that you read or write: whether to do the normal but slightly complex thing of recognising partition tables and a FAT filesystem, or roll your own mini filesystem, or treat the SD card as sequential block storage, or something in between.
One of the standard SD Card approaches for Acorn's BBC Micro is to have a rather minimal FAT implementation which just reads one file in the root directory: that file contains a number of disk images and then the disk filing system accesses those in the usual way by reading and writing sectors. In this way there's a minimal but useful compatibility with updating or saving the one big file on a PC. However, this approach builds on top of an existing disk file system, which you don't - yet - have.
