First, I like what you have done. Your work looks neat and tidy, too.
Rob makes a good point and his solution, if necessary (stronger straps piggybacking the fabric), is a good one.
Chas makes a good point, too. The idea I had... actually thought you were headed this way when I first started reading your post... was that the drop panel was going to ride on top of a ledger surrounding the cut out; that you would lift the floor up, twist it to the side some, then lower it thru the hole, sort of like a manhole cover that can fall in on itself (unlike a round one). A simple fabric strap handle (like the old boyscout belts) attached to the top of the panel would give purchase, but would not be obtrusive under foot.
The other option would be to fasten your toggle tabs to their shafts, have the shafts project up thru the floor and cap the shaft with another toggle so that you could release the toggles from above.
Just spit balling ideas like the typical Monday morning quarterback.
